Zokahn -Personal Blog-


Running the strongman for ALS


Virtualization is not cloud!

There is a distinct difference between virtualization as used in organizations for over a decade and the new evolution in IT called 'cloud'. In classic virtualization the focus is on the virtual machine itself. The compute resources, the virtualization hosts, are made to host multiple instances of a guest OS, possible connected to predefined networks and storage volumes. Manual or automated effort is needed to connect the VM to needed resources, which have to exist or need to be created up front for the VM can utilize it's resources.

In virtualization the VM can be tailored to the users exact specifications, a VM can be made to scale-up (add more compute resources to the single VM) to match a single application needs. A VM is typically nurtured and upgraded with minor and major software upgrades. In virtualization the SLA is typically targeted on the uptime and performance features of the actual VM.

In cloud the focus is extended, from the VM to the whole datacenter. All the datacenter facilities are presented to the user via self-service portals and/or via a application programming interface (API) these services consists of software defined entities that can be fully controlled by it's user without any intervention of IT staff.
The user can create their own datacenter environment based on a predefined budget, the datacenter is implemented automatically using on premises compute, network and storage resources or the user is (seamlessly) connected to resources remotely.

Cloud resources help the user to scale out (use smaller but more compute instances) as the virtual machines themselves cannot/should not be tailored to exact specifications. A cloud vm (instance) receives only the minor, security related, OS updates. Major release upgrades are performed by deleting a instance then redeploy a new instance based on a new OS image.

The SLA in cloud is typically targeted on the IT infrastructure, the ability to start a cloud instance (VM) and use it's datacenter facilities. The uptime of a specific VM should not matter and is normally not part of the Cloud SLA.


A bit about OpenStack processes

OpenStack can be overwhelming, it's big and complex. It might be hard to find the source of a problem, in this article i want to share some ways to get extra information about the processes and their settings. OpenStack software is developed in Python. Once a process starts it will show up in the process list. It starts to log information based on their configured logfile/dir location and verbosity. But what if the process fails to start? No messages, no hints on what went wrong.

Lets take the look at a typical OpenStack process. In this example we look at Glance but almost all OpenStack components follow the same principles. In a typical setup Glance will show up a few times in the process list. Running "ps -ef | grep glance" on a controller node will uncover the processes. We are interested in the 'CMD' field, the last part of the process line:

/usr/bin/python /usr/bin/glance-api --config-file /usr/share/glance/glance-api-dist.conf --config-file /etc/glance/glance-api.conf --verbose

This line gives a lot of information, in whole it gives you the complete command that the 'openstack-glance-api' init script uses to start the API process. This command invokes two config files, first glance-api-dist.conf then glance-api.conf. It also puts the process in verbose logging mode.

The config file part is good information, the glance-api-dist.conf holds the default and (most) mandatory configuration items and their values. Take a look at this file. These settings will apply if you set nothing in the glance-api.conf file. Anything set in the glance-api.conf file will override these settings, simply because the file is loaded after the dist file.

Let's say you are having problems getting a process started... You have run "service openstack-glance-api start" and nothing happens. The OS reports [OK] but glance-api is not listed in the process list. Checking the logging in /var/log/glance shows nothing. No new log entries... What would you do? You start the process by hand! Just paste this to the command line:

/usr/bin/python /usr/bin/glance-api --config-file /usr/share/glance/glance-api-dist.conf --config-file /etc/glance/glance-api.conf --verbose

This will result in the following output:

2013-11-28 10:01:07.306 2147 WARNING glance.store.base [-] Failed to configure store correctly: Store s3 could not be configured correctly. Reason: Could not find s3_store_host in configuration options. Disabling add method.
2013-11-28 10:01:07.315 2147 ERROR glance.store.swift [-] Could not find swift_store_auth_address in configuration options.
2013-11-28 10:01:07.315 2147 WARNING glance.store.base [-] Failed to configure store correctly: Store swift could not be configured correctly. Reason: Could not find swift_store_auth_address in configuration options. Disabling add method.

If there is anything preventing the Glance API to start, it will be listed in your output. This method also shows the exceptions that are not handled by the log facility developed by the OpenStack team. As said before this works for Glance but it will also work with most other OpenStack services.

Happy OpenStacking!


Should i use openstack-config?

The answer is yes, you really should. The tool is part of the openstack-utils package in the Red Hat world, this package is not always installed as a dependency. It has two options, you can either set or delete items in a config file for OpenStack. The Openstack Config files follow a (ini) structure like we have seen in the MS Windows world. A config file has sections, each sections has parameter and value pairs.

Most of the OpenStack components a have DEFAULT section, some have additional sections. I must compliment the developers for the persistence of this structure throughout the OpenStack project and all of it's components. Here is a example:

example or partial /etc/nova/nova.conf


We could change this file manually to change the verbose item to False by opening the file in a text editor of choice, change the line and save the file. This action will just work... But is pretty hard to document or script. You could use elaborate sed commands to accomplish what openstack-config can do for you without much thought. Changing the same with openstack-config is:

openstack-config --set /etc/nova/nova.conf DEFAULT verbose False

That's it. Running the command will set the config item from True to false. Completely delete a item value pair is just as easy:

openstack-config --del nova.conf DEFAULT verbose

Note: openstack-config will delete this item and probably break your setup...

You could add a bunch of these items into a script or documentation. Openstack-config can only handle one item per line and it will not check any of your section, item or value input. If you make a mistake, it will add your mistake to the file. It will just create a new section if you misspell the section by typing DFAULT. Same is true for the parameter name or value. It will accept anything. Openstack-config will check it's own options, either --set or --del and it will also check if the file exists. If the file does not exist it will fail to open it and present a Python Exception, this is a bit rough to the eyes. It will also set the exit code to 1 signaling the failure, of course it will exit with 0 is successful.

So... Should you use openstack-config to edit OpenStack config files? Yes... Make it a habit! It will help you automate things with ease and write clear documentation.


Persoonlijk record op de 10 ;-)


3 dagen vijf landen, deel 2


Drie dagen, vijf landen deel 1




Hardlopen: Nog eens 10K

Tagged as: Comments Off

Hardlopen… Heerlijk!