Rados-GW上的放置池

laurentbarbe

此测试的目的是将RadosGw Bucket映射到特定的Ceph池。例如，如果使用带有ssd的快速池和用于归档的低池…

1
2

   standard_bucket datas  --> .rgw.buckets        (default pool)
   specific_bucket datas  --> .rgw.buckets.custom

首先，我们创建一个池.rgw.buckets.custom，例如，使用一些特定的参数（crushmap中不同的尺寸和不同的规则集）

1
2
3
4
5
6
7
8

$ ceph osd pool create .rgw.buckets.custom 64 64
pool '.rgw.buckets.custom' created

$ ceph osd pool set .rgw.buckets.custom size 2
set pool 59 size to 2

$ ceph osd pool set .rgw.buckets.custom crush_ruleset 6
set pool 59 crush_ruleset to 6

然后，我们需要在区域映射和区域中配置特定的placement_targets。对于下一步，您需要有一个正在运行的rados-gw配置…

1
2

$ radosgw-admin region get > region.conf.json
$ vim region.conf.json            # Add an entry in placement_targets

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

{ "name": "default",
  "api_name": "",
  "is_master": "true",
  "endpoints": [],
  "master_zone": "",
  "zones": [
        { "name": "default",
          "endpoints": [],
          "log_meta": "false",
          "log_data": "false"}],
  "placement_targets": [
        { "name": "default-placement",
          "tags": []},
        { "name": "custom-placement",
          "tags": []}],
  "default_placement": "default-placement"}

1
2

$ radosgw-admin region set < region.conf.json
....

1
2

$ radosgw-admin zone get > zone.conf.json
$ vim zone.conf.json            # Add an entry in placement_pools with key "custom-placement"

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

{ "domain_root": ".rgw",
  "control_pool": ".rgw.control",
  "gc_pool": ".rgw.gc",
  "log_pool": ".log",
  "intent_log_pool": ".intent-log",
  "usage_log_pool": ".usage",
  "user_keys_pool": ".users",
  "user_email_pool": ".users.email",
  "user_swift_pool": ".users.swift",
  "user_uid_pool": ".users.uid",
  "system_key": { "access_key": "",
      "secret_key": ""},
  "placement_pools": [
        { "key": "default-placement",
          "val": { "index_pool": ".rgw.buckets.index",
              "data_pool": ".rgw.buckets",
              "data_extra_pool": ".rgw.buckets.extra"}},
        { "key": "custom-placement",
          "val": { "index_pool": ".rgw.buckets.index",
              "data_pool": ".rgw.buckets.custom",
              "data_extra_pool": ".rgw.buckets.extra"}}]}

1
2
3

$ radosgw-admin zone set <zone.conf.json
2014-11-25 18:03:23.894153 7f728c0f2780  0 couldn't find old data placement pools config, setting up new ones for the zone
.....

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

$ radosgw-admin regionmap update
{ "regions": [
        { "key": "default",
          "val": { "name": "default",
              "api_name": "",
              "is_master": "true",
              "endpoints": [],
              "master_zone": "",
              "zones": [
                    { "name": "default",
                      "endpoints": [],
                      "log_meta": "false",
                      "log_data": "false"}],
              "placement_targets": [
                    { "name": "custom-placement",
                      "tags": []},
                    { "name": "default-placement",
                      "tags": []}],
              "default_placement": "default-placement"}}],
  "master_region": "default",
  "bucket_quota": { "enabled": false,
      "max_size_kb": -1,
      "max_objects": -1},
  "user_quota": { "enabled": false,
      "max_size_kb": -1,
      "max_objects": -1}}

$ /etc/init.d/radosgw reload
Reloading ...

要为RadosGW配置s3cmd，您可以查看这里：http://lollyrock.com/articles/s3cmd-with-radosgw/

现在我们可以测试bucket创建

1
2
3
4
5
6
7
8
9

$ s3cmd mb s3://custombucket --bucket-location=custom-placement
Bucket 'custombucket' created

$ touch "file_on_custom_pool"

$ s3cmd put file_on_custom_pool s3://custombucket
WARNING: Module python-magic is not available. Guessing MIME types based on file extensions.
file_on_custom_pool -> s3://custombucket/file_on_custom_pool  [1 of 1]
 0 of 0     0% in    0s     0.00 B/s  done

验证

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

$ radosgw-admin bucket stats --bucket=custombucket
{ "bucket": "custombucket",
  "pool": ".rgw.buckets.custom",
  "index_pool": ".rgw.buckets.index",
  "id": "default.240909.1",
  "marker": "default.240909.1",
  "owner": "testuser",
  "ver": 1,
  "master_ver": 0,
  "mtime": 1417016078,
  "max_marker": "",
  "usage": {},
  "bucket_quota": { "enabled": false,
      "max_size_kb": -1,
      "max_objects": -1}}

池变量已设置为“.rgw.buckets.custom”。

1
2

$ rados -p .rgw.buckets.custom ls
default.241071.1_file_on_custom_pool

就在这里！

数据放置池的定义顺序如下

  1. 来自请求（“bucket location”）
  2. 来自用户（“default_placement”：请参阅radosgw-admin metadata get user:<uid>）
  3. 来自区域映射（“default_placement”）