一、简介
Openstack块存储服务Cinder主要为实例提供块设备存储服务,类似可热插拔的移动硬盘的功能。存储的分配和消耗是由块存储驱动器,或者多后端配置的驱动器决定的。目前有很多驱动程序可用,如NAS/SAN,NFS,ISCSI,Ceph等。一般情况下,块服务API和调度器服务运行在控制节点上。块存储服务器可以运行在控制节点、计算节点或单独的存储节点,具体取决于使用的驱动。
Cinder插件驱动程序分为基于软件系统和硬件系统的两类,具体分类可参考下图(图片来源:http://www.cnblogs.com/sammyliu/p/4219974.html):
Cinder组件由三部分组成:
- cinder-api 组件:负责向外提供Cinder REST API;
- cinder-scheduler 组件:根据调度算法,负责分配存储资源;
- cinder-volume 组件:负责封装driver,不同的driver负责控制不同的后端存储
二、部署脚本
Cinder块存储配置在三个控制节点上,此处后端存储对接Ceph统一存储,由于Ceph在后面配置,脚本运行完成后执行cinder service-list后cinder-volume服务会处于down状态,配置脚本install-configure-cinder.sh如下:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 |
#!/bin/sh . ../0-set-config.sh ./style/print-split.sh "Cinder Installation" ### [所有控制节点] 安装配置 ./pssh-exe C "yum install -y openstack-cinder" ### [所有控制节点]配置配置cinder组件,/etc/nova/nova.conf文件 for ((i=0; i<${#controller_map[@]}; i+=1)); do name=${controller_name[$i]}; ip=${controller_map[$name]}; . style/print-info.sh "Openstack configure in $name" ssh root@$ip /bin/bash << EOF openstack-config --set /etc/cinder/cinder.conf database connection mysql+pymysql://cinder:$password@$virtual_ip/cinder openstack-config --set /etc/cinder/cinder.conf database max_retries -1 openstack-config --set /etc/cinder/cinder.conf DEFAULT auth_strategy keystone openstack-config --set /etc/cinder/cinder.conf keystone_authtoken auth_uri http://$virtual_ip:5000 openstack-config --set /etc/cinder/cinder.conf keystone_authtoken auth_url http://$virtual_ip:35357 openstack-config --set /etc/cinder/cinder.conf keystone_authtoken memcached_servers controller01:11211,controller02:11211,controller03:11211 openstack-config --set /etc/cinder/cinder.conf keystone_authtoken auth_type password openstack-config --set /etc/cinder/cinder.conf keystone_authtoken project_domain_name default openstack-config --set /etc/cinder/cinder.conf keystone_authtoken user_domain_name default openstack-config --set /etc/cinder/cinder.conf keystone_authtoken project_name service openstack-config --set /etc/cinder/cinder.conf keystone_authtoken username cinder openstack-config --set /etc/cinder/cinder.conf keystone_authtoken password $password openstack-config --set /etc/cinder/cinder.conf DEFAULT rpc_backend rabbit openstack-config --set /etc/cinder/cinder.conf oslo_messaging_rabbit rabbit_hosts controller01:5672,controller02:5672,controller03:5672 openstack-config --set /etc/cinder/cinder.conf oslo_messaging_rabbit rabbit_ha_queues true openstack-config --set /etc/cinder/cinder.conf oslo_messaging_rabbit rabbit_retry_interval 1 openstack-config --set /etc/cinder/cinder.conf oslo_messaging_rabbit rabbit_retry_backoff 2 openstack-config --set /etc/cinder/cinder.conf oslo_messaging_rabbit rabbit_max_retries 0 openstack-config --set /etc/cinder/cinder.conf oslo_messaging_rabbit rabbit_durable_queues true openstack-config --set /etc/cinder/cinder.conf oslo_messaging_rabbit rabbit_userid openstack openstack-config --set /etc/cinder/cinder.conf oslo_messaging_rabbit rabbit_password $password openstack-config --set /etc/cinder/cinder.conf DEFAULT control_exchange cinder openstack-config --set /etc/cinder/cinder.conf DEFAULT osapi_volume_listen $ip openstack-config --set /etc/cinder/cinder.conf DEFAULT my_ip $ip openstack-config --set /etc/cinder/cinder.conf oslo_concurrency lock_path /var/lib/cinder/tmp openstack-config --set /etc/cinder/cinder.conf DEFAULT glance_api_servers http://$virtual_ip:9292 openstack-config --set /etc/glance/glance-api.conf DEFAULT show_image_direct_url True openstack-config --set /etc/glance/glance-api.conf glance_store stores rbd openstack-config --set /etc/glance/glance-api.conf glance_store default_store rbd openstack-config --del /etc/glance/glance-api.conf glance_store filesystem_store_datadir openstack-config --set /etc/glance/glance-api.conf glance_store rbd_store_pool images openstack-config --set /etc/glance/glance-api.conf glance_store rbd_store_user glance openstack-config --set /etc/glance/glance-api.conf glance_store rbd_store_ceph_conf /etc/ceph/ceph.conf openstack-config --set /etc/glance/glance-api.conf glance_store rbd_store_chunk_size 8 openstack-config --set /etc/glance/glance-api.conf paste_deploy flavor keystone openstack-config --set /etc/cinder/cinder.conf DEFAULT enabled_backends ceph openstack-config --set /etc/cinder/cinder.conf DEFAULT glance_api_version 2 openstack-config --set /etc/cinder/cinder.conf ceph volume_driver cinder.volume.drivers.rbd.RBDDriver openstack-config --set /etc/cinder/cinder.conf ceph rbd_pool volumes openstack-config --set /etc/cinder/cinder.conf ceph rbd_ceph_conf /etc/ceph/ceph.conf openstack-config --set /etc/cinder/cinder.conf ceph rbd_flatten_volume_from_snapshot false openstack-config --set /etc/cinder/cinder.conf ceph rbd_max_clone_depth 5 openstack-config --set /etc/cinder/cinder.conf ceph rbd_store_chunk_size 4 openstack-config --set /etc/cinder/cinder.conf ceph rados_connect_timeout -1 openstack-config --set /etc/cinder/cinder.conf ceph glance_api_version 2 openstack-config --set /etc/cinder/cinder.conf ceph rbd_user cinder openstack-config --set /etc/cinder/cinder.conf ceph rbd_secret_uuid 032198f4-b815-4254-9de2-185f935bd7de openstack-config --set /etc/cinder/cinder.conf DEFAULT backup_driver cinder.backup.drivers.ceph openstack-config --set /etc/cinder/cinder.conf DEFAULT backup_ceph_conf /etc/ceph/ceph.conf openstack-config --set /etc/cinder/cinder.conf DEFAULT backup_ceph_user cinder-backup openstack-config --set /etc/cinder/cinder.conf DEFAULT backup_ceph_chunk_size 134217728 openstack-config --set /etc/cinder/cinder.conf DEFAULT backup_ceph_pool backups openstack-config --set /etc/cinder/cinder.conf DEFAULT backup_ceph_stripe_unit 0 openstack-config --set /etc/cinder/cinder.conf DEFAULT backup_ceph_stripe_count 0 openstack-config --set /etc/cinder/cinder.conf DEFAULT restore_discard_excess_bytes true openstack-config --set /etc/cinder/cinder.conf DEFAULT host cinder-cluster service openstack-glance-api restart service openstack-cinder-volume restart service openstack-cinder-backup restart EOF done; ### [任一节点]创建数据库 mysql -uroot -p$password_galera_root -h $virtual_ip -e "CREATE DATABASE cinder; GRANT ALL PRIVILEGES ON cinder.* TO 'cinder'@'localhost' \ IDENTIFIED BY '"$password"'; GRANT ALL PRIVILEGES ON cinder.* TO 'cinder'@'controller01' \ IDENTIFIED BY '"$password"'; GRANT ALL PRIVILEGES ON cinder.* TO 'cinder'@'%' \ IDENTIFIED BY '"$password"'; FLUSH PRIVILEGES;" ##### generate haproxy.cfg . ./1-gen-haproxy-cfg.sh cinder ### [任一节点]创建用户等 . /root/keystonerc_admin ### [任一节点]创建用户等 openstack user create --domain default --password $password cinder openstack role add --project service --user cinder admin openstack service create --name cinder --description "OpenStack Block Storage" volume openstack service create --name cinderv2 --description "OpenStack Block Storage" volumev2 ###创建cinder服务的API endpoints openstack endpoint create --region RegionOne volume public http://$virtual_ip:8776/v1/%\(tenant_id\)s openstack endpoint create --region RegionOne volume internal http://$virtual_ip:8776/v1/%\(tenant_id\)s openstack endpoint create --region RegionOne volume admin http://$virtual_ip:8776/v1/%\(tenant_id\)s openstack endpoint create --region RegionOne volumev2 public http://$virtual_ip:8776/v2/%\(tenant_id\)s openstack endpoint create --region RegionOne volumev2 internal http://$virtual_ip:8776/v2/%\(tenant_id\)s openstack endpoint create --region RegionOne volumev2 admin http://$virtual_ip:8776/v2/%\(tenant_id\)s ### [任一节点]生成数据库 su -s /bin/sh -c "cinder-manage db sync" cinder ### [任一节点]添加pacemaker资源 pcs resource create openstack-cinder-api systemd:openstack-cinder-api --clone interleave=true pcs resource create openstack-cinder-scheduler systemd:openstack-cinder-scheduler --clone interleave=true pcs resource create openstack-cinder-volume systemd:openstack-cinder-volume pcs constraint order start openstack-keystone-clone then openstack-cinder-api-clone pcs constraint order start openstack-cinder-api-clone then openstack-cinder-scheduler-clone pcs constraint colocation add openstack-cinder-scheduler-clone with openstack-cinder-api-clone pcs constraint order start openstack-cinder-scheduler-clone then openstack-cinder-volume pcs constraint colocation add openstack-cinder-volume with openstack-cinder-scheduler-clone pcs resource op add openstack-cinder-api start timeout=300 pcs resource op add openstack-cinder-api stop timeout=300 pcs resource op add openstack-cinder-scheduler start timeout=300 pcs resource op add openstack-cinder-scheduler stop timeout=300 pcs resource op add openstack-cinder-volume start timeout=300 pcs resource op add openstack-cinder-volume stop timeout=300 . restart-pcs-cluster.sh cinder service-list |
三、参考文档
https://docs.openstack.org/mitaka/install-guide-rdo/cinder.html
http://docs.ceph.com/docs/master/rbd/rbd-openstack/
https://docs.openstack.org/ha-guide/storage-ha.html
http://www.cnblogs.com/sammyliu/p/4219974.html
四、源码
脚本源码:https://github.com/zjmeixinyanzhi/Openstack-HA-Install-Shells
五、系列文章
Openstack云平台脚本部署之Galera高可用集群配置(二)
Openstack云平台脚本部署之RabbitMQ高可用集群部署(三)
Openstack云平台脚本部署之Memcached配置(五)
Openstack云平台脚本部署之Keystone认证服务配置(六)
Openstack云平台脚本部署之Glance镜像服务配置(七)
Openstack云平台脚本部署之Nova计算服务配置(八)
Openstack云平台脚本部署之Neutron网络服务配置(九)
Openstack云平台脚本部署之Dashboard配置(十)
Openstack云平台脚本部署之Cinder块存储服务配置(十一)
Openstack云平台脚本部署之Ceilometer数据收集服务配置(十二)
Openstack云平台脚本部署之Aodh告警服务配置(十三)
三节点主控+ceph部署完成
测试场景1: 强制关闭现有主节点controller1
发现所有的cinder-volume服务都down了
| cinder-volume | controller1@ceph | nova | enabled | down | 2017-09-14T07:02:13.000000 | – |
| cinder-volume | controller2@ceph | nova | enabled | down | 2017-09-14T07:02:12.000000 | – |
| cinder-volume | controller3@ceph | nova | enabled | down | 2017-09-14T07:02:05.000000 | – |
求解!
感谢
cinder-volume中配置你可能配错了,就是所有节点host设置是一致的,比如:openstack-config –set /etc/cinder/cinder.conf DEFAULT host cinder-cluster,然后执行验证时,cinder-volume服务状态只有一条而不是三条,如下:
| cinder-volume | cinder-cluster@ceph | nova | enabled | up | 2017-09-16T09:04:19.000000 | – |。