Main Page/Gretting Started With Openstack Code

Category:development
=Getting started understanding the code and objects=
== Setup a devstack environment ==
it also works using centos:
https://docs.openstack.org/devstack/latest/guides/single-machine.html
I had to add HOST_IP= with my local ip on my local.conf
Les try to use the python client to understand the code. Schedulers should be easy.
== Test units are simple code snippets ==
So sitep-packages/nova/scheduler/filters doesn’t give any clue on how these should be used… but found the tests units that are a good example (I think they can be found in a separate package, at least for fedora)
:
/home/ebarrera/PycharmProjects/nova/nova/tests/unit/scheduler/filters
import mock
from nova import objects
from nova.scheduler.filters import disk_filter
from nova import test
from nova.tests.unit.scheduler import fakes
class TestDiskFilter(test.NoDBTestCase):
def test_disk_filter_passes(self):
filt_cls = disk_filter.DiskFilter()
spec_obj = objects.RequestSpec(
 flavor=objects.Flavor(root_gb=1, ephemeral_gb=1, swap=512));
host = fakes.FakeHostState('host1', 'node1',
 {'free_disk_mb': 11 * 1024, 'total_usable_disk_gb': 13,
 'disk_allocation_ratio': 1.0})
self.assertTrue(filt_cls.host_passes(host, spec_obj))
This give me a great clue of the parameters I shall pass to the functions,
flavor=objects.Flavor(root_gb=1, ephemeral_gb=1, swap=512))
Looking at nova/objects.flavor.py 
fields = {
‘id’: fields.IntegerField(),
‘name’: fields.StringField(nullable=True),
‘memory_mb’: fields.IntegerField(),
‘vcpus’: fields.IntegerField(),
‘root_gb’: fields.IntegerField(),
‘ephemeral_gb’: fields.IntegerField(),
……
}
Executing this give me error… doing >>> dir(object) I see almost nothing
I see.. I have to do an .. first
 >>> objects.register_all() 
Now this works
 >>> flavor=objects.Flavor(root_gb=1, ephemeral_gb=1, swap=512)
 
 print flavor 
 
Flavor(created_at=<?>,deleted=<?>,deleted_at=<?>,description=<?>,disabled=<?>,ephemeral_gb=1,extra_specs=<?>,flavorid=<?>,id=<?>,is_public=<?>,memory_mb=<?>,name=<?>,projects=<?>,root_gb=1,rxtx_factor=<?>,swap=512,updated_at=<?>,vcpu_weight=<?>,vcpus=<?>)
== Time for HostState ==
Then I do >>> help(object) and a great text appears, I see it talks about a function register_all()
flavor=objects.register_all()
Then if I do a >>> dir(object.Flacor) I see much more things… Now executing that commend works:
flavor=objects.Flavor(root_gb=1, ephemeral_gb=1, swap=512)
The previous code line was enclosed in another funcion:
spec_obj = objects.RequestSpec(
flavor=objects.Flavor(root_gb=1, ephemeral_gb=1, swap=512))
Lets take a look at objects/request_spec.py
https://specs.openstack.org/openstack/nova-specs/specs/kilo/approved/request-spec-object.html
Is this an object where we can plug more objects ?
fields = {
‘id’: fields.IntegerField(),
…
‘flavor’: fields.ObjectField(‘Flavor’, nullable=False),
No idea, lets asume yes and continue with the next line….
host = fakes.FakeHostState(‘host1’, ‘node1’,
{‘free_disk_mb’: 11 * 1024, ‘total_usable_disk_gb’: 13,
‘disk_allocation_ratio’: 1.0})
We have to take a look of the Fakes class, lets see what it does… it is in the tests directories… so vi fakes.py
Very interesting … we can see here how a computeNode object is defined.. The functions seems to return the Hoststate information for node1
<syntaxhighlight lang="python">
>>> from nova import objects
>>> from nova.scheduler.filters import disk_filter
>>> from nova import test
>>> from nova.tests.unit.scheduler import fakes
>>>
>>> filt_cls = disk_filter.DiskFilter()
>>> spec_obj = objects.RequestSpec(
… flavor=objects.Flavor(root_gb=1, ephemeral_gb=1, swap=512))
>>>
>>> print spec_obj.flavor.root_gb
1
>>> host = fakes.FakeHostState(‘host1’, ‘node1’,
… {‘free_disk_mb’: 11 * 1024, ‘total_usable_disk_gb’: 13,
… ‘disk_allocation_ratio’: 1.0})
>>> filt_cls.host_passes(host, spec_obj)
True
>>>
You can put the previous code on a file and run it -i parameter, you will get a the python prompt an you will be able to query the classes and variables
# python -i disck_scheudler.py
To troubleshoot scheduling issues It would be nice to obtain the hoststats of customer’s envoronment.. lets try to do a script that does it
…
I see context is used everywhere so bertter understand what is it
https://docs.openstack.org/oslo.context/latest/user/usage.html
Create this file:
cat context.py
from oslo_config import cfg
from oslo_context import context
from oslo_log import log as logging
CONF = cfg.CONF
DOMAIN = “demo”
logging.register_options(CONF)
logging.setup(CONF, DOMAIN)
LOG = logging.getLogger(__name__)
LOG.info(“Message without context”)
# ids in Openstack are 32 characters long
# For readability a shorter id value is used
context.RequestContext(user=’6ce90b4d’,
tenant=’d6134462′,
project_domain=’a6b9360e’)
LOG.info(“Message with context”)
$ python -i context.py
>> myctxt=context.RequestContext(user=’6ce90b4d’,
… tenant=’d6134462′,
… project_domain=’a6b9360e’)
>>> print myctxt
>>> dir(myctxt)
[‘FROM_DICT_EXTRA_KEYS’, ‘__class__’, ‘__delattr__’, ‘__dict__’, ‘__doc__’, ‘__format__’, ‘__getattribute__’, ‘__hash__’, ‘__init__’, ‘__module__’, ‘__new__’, ‘__reduce__’, ‘__reduce_ex__’, ‘__repr__’, ‘__setattr__’, ‘__sizeof__’, ‘__str__’, ‘__subclasshook__’, ‘__weakref__’, ‘_domain_id’, ‘_project_domain_id’, ‘_project_id’, ‘_user_domain_id’, ‘_user_id’, ‘auth_token’, ‘domain’, ‘domain_id’, ‘domain_name’, ‘from_dict’, ‘from_environ’, ‘get_logging_values’, ‘global_id’, ‘global_request_id’, ‘is_admin’, ‘is_admin_project’, ‘project_domain’, ‘project_domain_id’, ‘project_domain_name’, ‘project_id’, ‘project_name’, ‘read_only’, ‘request_id’, ‘resource_uuid’, ‘roles’, ‘service_project_domain_id’, ‘service_project_domain_name’, ‘service_project_id’, ‘service_project_name’, ‘service_roles’, ‘service_token’, ‘service_user_domain_id’, ‘service_user_domain_name’, ‘service_user_id’, ‘service_user_name’, ‘show_deleted’, ‘tenant’, ‘to_dict’, ‘to_policy_values’, ‘update_store’, ‘user’, ‘user_domain’, ‘user_domain_id’, ‘user_domain_name’, ‘user_id’, ‘user_idt_format’, ‘user_name’]
Note all the information we can fit in the context
https://docs.openstack.org/nova/mitaka/api/nova.context.html
$ cat context.py
from oslo_config import cfg
from oslo_context import context
from oslo_log import log as logging
from nova.common import config
from nova import version
from oslo_log import log
from nova import context as nova_context
from nova.scheduler import host_manager
from nova import objects
objects.register_all()
CONF = cfg.CONF
DOMAIN = “demo”
logging.register_options(CONF)
logging.setup(CONF, DOMAIN)
LOG = logging.getLogger(__name__)
LOG.info(“Message without context”)
# ids in Openstack are 32 characters long
# For readability a shorter id value is used
osloctx=context.RequestContext(user=’6ce90b4d’,
tenant=’d6134462′,
project_domain=’a6b9360e’)
LOG.info(“Message with context”)
CONF(“”, project=’nova’, version=version.version_string(), default_config_files=None)
manager=host_manager.HostManager()
ctx-nova_context.RequestContext()
print manager.get_all_host_states(ctx)
2018-03-02 13:05:41.858 969039 ERROR demo File “/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py”, line 393, in connect
2018-03-02 13:05:41.858 969039 ERROR demo return self.dbapi.connect(*cargs, **cparams)
2018-03-02 13:05:41.858 969039 ERROR demo File “/usr/lib64/python2.7/site-packages/MySQLdb/__init__.py”, line 81, in Connect
2018-03-02 13:05:41.858 969039 ERROR demo return Connection(*args, **kwargs)
2018-03-02 13:05:41.858 969039 ERROR demo File “/usr/lib64/python2.7/site-packages/MySQLdb/connections.py”, line 193, in __init__
2018-03-02 13:05:41.858 969039 ERROR demo super(Connection, self).__init__(*args, **kwargs2)
2018-03-02 13:05:41.858 969039 ERROR demo OperationalError: (_mysql_exceptions.OperationalError) (1045, “Access denied for user ‘nova’@’localhost’ (using password: YES)”)
2018-03-02 13:05:41.858 969039 ERROR demo
* NOTE: the previous errors are because I was running my tests on a OSP12 and /etc/nova.conf is not the correct file to check, nova.conf is inside the container, use a devstack env better
>>> print manager.host_state_map
{(u’centos-7-newton’, u’centos-7-newton.default’): (centos-7-newton, centos-7-newton.default) ram: 6679MB disk: 5120MB io_ops: 0 instances: 0}
It worked…
>>> for i in manager.get_all_host_states(ctx):
… print i
…
(centos-7-newton, centos-7-newton.default) ram: 6679MB disk: 5120MB io_ops: 0 instances: 0
>>> print manager.get_all_host_states(ctx).next()
(centos-7-newton, centos-7-newton.default) ram: 6679MB disk: 5120MB io_ops: 0 instances: 0
>>> print CONF.items()
2018-03-02 18:08:14.569 14388 WARNING oslo_config.cfg [req-6d9b01ef-8595-4756-8a77-d580004da51f – – – – -] Option “compute_manager” from group “DEFAULT” is deprecated for removal. Its value may be silently ignored in the future.
[(‘bandwidth_poll_interval’, 600), (‘default_floating_pool’, ‘public’), (‘metadata_listen_port’, 8775), (‘num_networks’, 1), (‘image_
….
….
This works on my centos-newton but fails in openstack03.controller-0
>>> print CONF.report_interval
2018-03-02 18:11:20.180 11235 CRITICAL demo [req-16f698be-3dd2-4cbc-91f4-46518c33a982 6ce90b4d d6134462 – – a6b9360e] NoSuchOptError: no such option in group DEFAULT: report_interval
2018-03-02 18:11:20.180 11235 ERROR demo Traceback (most recent call last):
2018-03-02 18:11:20.180 11235 ERROR demo File “”, line 1, in
2018-03-02 18:11:20.180 11235 ERROR demo File “/usr/lib/python2.7/site-packages/oslo_config/cfg.py”, line 2187, in __getattr__
2018-03-02 18:11:20.180 11235 ERROR demo raise NoSuchOptError(name)
2018-03-02 18:11:20.180 11235 ERROR demo NoSuchOptError: no such option in group DEFAULT: report_interval
2018-03-02 18:11:20.180 11235 ERROR demo
>>>
2018-03-02 18:13:30.510 11235 CRITICAL demo [req-16f698be-3dd2-4cbc-91f4-46518c33a982 6ce90b4d d6134462 – – a6b9360e] KeyboardInterrupt
2018-03-02 18:13:30.510 11235 ERROR demo KeyboardInterrupt
2018-03-02 18:13:30.510 11235 ERROR demo
Looking at https://review.openstack.org/#/c/468849/
Seems that nova.common is not installed in the server, the script also complains importing it
It would be better if I learn without using nova.common