一、简介
今天在现场调整虚拟机大小时出现错误，查看了日志和网上的解决方案，问题是计算节点之间nova用户之间需要配置SSH，所以配置完成即可。
日志出错内容如下：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 |
2017-05-06 13:13:23.383 156583 INFO nova.compute.manager [req-102fbe46-d741-4f74-a966-e81662a71c80 0733a705b68d426095938fd615b0db61 2dab6332fbfe4d9d8d19080d596eba95 - - -] [instance: d214367d-a7ca-46a0-95a9-1e42420a1eb5] Setting instance back to ACTIVE after: Instance rollback performed due to: Resize error: not able to execute ssh command: Unexpected error while running command. Command: ssh 172.20.103.7 mkdir -p /var/lib/nova/instances/d214367d-a7ca-46a0-95a9-1e42420a1eb5 Exit code: 255 Stdout: u'' Stderr: u"Warning: Permanently added '172.20.103.7' (ECDSA) to the list of known hosts.\r\nPermission denied, please try again.\r\nPermission denied, please try again.\r\nPermission denied (publickey,gssapi-keyex,gssapi-with-mic,password).\r\n" 2017-05-06 13:13:23.711 156583 INFO nova.compute.manager [req-102fbe46-d741-4f74-a966-e81662a71c80 0733a705b68d426095938fd615b0db61 2dab6332fbfe4d9d8d19080d596eba95 - - -] [instance: d214367d-a7ca-46a0-95a9-1e42420a1eb5] Successfully reverted task state from None on failure for instance. 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher [req-102fbe46-d741-4f74-a966-e81662a71c80 0733a705b68d426095938fd615b0db61 2dab6332fbfe4d9d8d19080d596eba95 - - -] Exception during message handling: Resize error: not able to execute ssh command: Unexpected error while running command. Command: ssh 172.20.103.7 mkdir -p /var/lib/nova/instances/d214367d-a7ca-46a0-95a9-1e42420a1eb5 Exit code: 255 Stdout: u'' Stderr: u"Warning: Permanently added '172.20.103.7' (ECDSA) to the list of known hosts.\r\nPermission denied, please try again.\r\nPermission denied, please try again.\r\nPermission denied (publickey,gssapi-keyex,gssapi-with-mic,password).\r\n" 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last): 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 138, in _dispatch_and_reply 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher incoming.message)) 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 183, in _dispatch 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args) 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 127, in _do_dispatch 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args) 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/exception.py", line 110, in wrapped 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher payload) 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher self.force_reraise() …… 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher migration.instance_uuid, exc_info=True) 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher self.force_reraise() 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 293, in decorated_function 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs) 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 387, in decorated_function 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher kwargs['instance'], e, sys.exc_info()) 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher self.force_reraise() 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb) 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 375, in decorated_function 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher return function(self, context, *args, **kwargs) 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 3932, in resize_instance 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher self.instance_events.clear_events_for_instance(instance) 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib64/python2.7/contextlib.py", line 35, in __exit__ 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher self.gen.throw(type, value, traceback) 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 6616, in _error_out_instance_on_exception 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher raise error.inner_exception 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher ResizeError: Resize error: not able to execute ssh command: Unexpected error while running command. 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher Command: ssh 172.20.103.7 mkdir -p /var/lib/nova/instances/d214367d-a7ca-46a0-95a9-1e42420a1eb5 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher Exit code: 255 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher Stdout: u'' 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher Stderr: u"Warning: Permanently added '172.20.103.7' (ECDSA) to the list of known hosts.\r\nPermission denied, please try again.\r\nPermission denied, please try again.\r\nPermission denied (publickey,gssapi-keyex,gssapi-with-mic,password).\r\n" 2017-05-06 13:13:23.722 156583 ERROR oslo_messaging.rpc.dispatcher |
二、处理步骤
对nova用户进行配置。
1、取消禁止登录权限
在所有计算节点上执行以下操作
|
1 |
# usermod -s /bin/bash nova |
检查修改，确保nova用户具备shell执行权限：
|
1 2 |
# cat /etc/passwd | grep nova nova:x:162:162:OpenStack Nova Daemons:/var/lib/nova:/bin/bash |
2、配置SSH
在任意一个计算节点上，用nova用户登录，创建密钥
|
1 2 |
# su - nova $ ssh-keygen -t rsa |
在该节点上，配置nova用户的SSH配置，不执行主机密钥验证，
|
1 2 3 4 5 |
$ cat << EOF > ~/.ssh/config Host * StrictHostKeyChecking no UserKnownHostsFile=/dev/null EOF |
拷贝id_rsa.pub为authorized_keys，并修改权限
|
1 2 |
cat ~/.ssh/id_rsa.pub > .ssh/authorized_keys chmod 600 .ssh/authorized_keys |
3、将配置拷贝至其他所有计算节点
将上面节点的nova根目录下.ssh目录拷贝其他所有计算节点上，并修改文件所属权限。
|
1 2 |
# scp -R /var/lib/nova/.ssh/ computeXXX:/var/lib/nova/ # ssh computeXXX chown nova:nova -R /var/lib/nova/.ssh/ |
在任意节点验证，到其他所有计算节点可否SSH登录
|
1 2 |
# su - nova $ ssh nova@172.20.103.7 |
4、重试resize，即可完成，发现resize完成后，虚拟机迁移到目标计算节点上。
三、疑问
以上的操作需要配置计算点间无密码访问，但是那么多计算节点相互访问配置也比较麻烦啊，有提到通过修改nova配置文件，允许在同一台服务器上进行resize和migrate操作配置项，如下：
|
1 2 3 |
# vim /etc/nova/nova.conf allow_resize_to_same_host=true allow_migrate_to_same_host=true |
但在计算节点配置后，重启nova-compute服务，重试resize没有效果。后来才发现，这仅适用于计算节点只有一台的测试环境，多计算节点的生产环境并不适用，仍需要ssh到其他计算节点。
四、一键配置脚本
备注：脚本这里用到了个人封装的pssh-exe和scp-exe两个工具脚本，作用分别为在其他节点执行命令以及拷贝到其他节点上，具体封装参考《Openstack云平台脚本部署之基础环境配置（一）》博客。
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
#!/bin/sh ./pssh-exe A "usermod -s /bin/bash nova" su - nova -c "ssh-keygen -t rsa" cat << EOF > /var/lib/nova/.ssh/config Host * StrictHostKeyChecking no UserKnownHostsFile=/dev/null EOF cat /var/lib/nova/.ssh/id_rsa.pub > /var/lib/nova/.ssh/authorized_keys chmod 600 /var/lib/nova/.ssh/authorized_keys ./scp-exe H /var/lib/nova/.ssh/ /var/lib/nova/ ./pssh-exe H "chown nova:nova -R /var/lib/nova/.ssh/" ### Check SSH su - nova -c "ssh nova@compute02 cd ~ && pwd" |
五、参考文档
http://paste.openstack.org/show/220970/
http://cloudbps.com/2016/01/11/openstack-resize-ssh/
https://ask.openstack.org/en/question/63999/resize-instance-ssh-warning/
http://www.adminz.in/2015/04/enabling-instance-resizing-in-openstack.html
codemore code
~~~~