Repository: fedora-infra/bodhi

Issue:
Work around missing repodata.xml
Sometimes the previous run of the Bodhi masher leaves some directories with missing repodata.xml files (See #887). That issue is about stopping that from happening in the future, but it is a bit mysterious and may take some time to identify its cause.

In the meantime, we can make Bodhi more robust. In subsequent runs of the masher, there is a test for the existence of the folder that should contain a repodata.xml file and if that folder is present Bodhi assumes that the file is present as well. This assumption is false, and it causes Bodhi to blow up later.

We should adjust that test to test for the existence of the file instead of the folder. This will make Bodhi more robust for releng, and should be a pretty easy change as well.



Discussion:
can i work on this

@amitsin6h sure, that would be great!

so how can i start. excited \o/

Hey @amitsin6h! The easiest way to get start with Bodhi development is to use our Vagrant setup:

https://github.com/fedora-infra/bodhi/blob/develop/README.rst#vagrant

I'm not actually too familiar with the specific code that this ticket is about, so I'm not sure I can give you very specific pointers for it. I marked it easy fix because once you find that if statement (wherever it is), it's likely just a matter of adjusting the check to test for the existence of the file rather than the directory that contains it.

hello,can I also work on it?

Hello @himanshuraja! This is probably going to be a pretty small change, but if you want to coordinate with @amitsin6h go for it!

Yes but I am new to this , can you please help me out in starting ? Thank you . 

Sent from my iPhone

> On 10-Sep-2016, at 3:22 AM, Randy Barlow notifications@github.com wrote:
> 
> Hello @himanshuraja! This is probably going to be a pretty small change, but if you want to coordinate with @amitsin6h go for it!
> 
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub, or mute the thread.

@amitsin6h @himanshuraja You probably need to check https://github.com/fedora-infra/bodhi/blob/develop/bodhi/server/metadata.py
https://github.com/fedora-infra/bodhi/blob/develop/bodhi/server/consumers/masher.py#L770-L816

Might want to look at the testcases also?
https://github.com/fedora-infra/bodhi/blob/develop/bodhi/tests/server/test_masher.py
https://github.com/fedora-infra/bodhi/blob/develop/bodhi/tests/server/test_metadata.py

thanks a lot @trishnaguha .

https://download.fedoraproject.org/pub/fedora/linux/updates/%s/%s/repodata/repomd.xml
is giving error cause .

@himanshuraja It would be better to come over on IRC for any issues you are facing. 

Hi,
I am new to contributing,can you please share the channel where I can log
on.
Thanks.
Regards,
Himanshu Sharma
dgplug,Acadgild

On Sat, Sep 10, 2016 at 5:46 PM, Sayan Chowdhury notifications@github.com
wrote:

> @himanshuraja https://github.com/himanshuraja It would be better to
> come over on IRC for any issues you are facing.
> 
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> https://github.com/fedora-infra/bodhi/issues/908#issuecomment-246108634,
> or mute the thread
> https://github.com/notifications/unsubscribe-auth/AJAt1tgb23qZBtcVvSBojSgdY1FNxk3-ks5qop-PgaJpZM4J2H5-
> .

The channel is `#fedora-apps` on Freenode.

I really appreciate it,thank you. 

am having problem in setting up bodhi development am setting up using virtualenv

@amitsin6h Can you describe the problem further? Maybe include the error message or a link to a log?

@k3rn3l3rr0r @trishnaguha @bowlofeggs @sayanchowdhury     http://paste.fedoraproject.org/427071/36593701/

@amitsin6h @bowlofeggs I can't test it right now on virtualenv host, but it seems like a deps/path problem coming from link_system_libs() in the bootstrap.py ... This conversation should be moved in new, separate issue.

@amitsin6h it looks to me like maybe you don't have a virtuenv enabled.

@bowlofeggs how should i enable it

@amitsin6h I'm not sure, but it does sound like there might be a problem in the virtualenv setup. I agree with @k3rn3l3rr0r, and recommend filing a separate issue as we don't want to fill this ticket up with non-related conversation.


Resolving patch:
diff --git a/bodhi/server/metadata.py b/bodhi/server/metadata.py
--- a/bodhi/server/metadata.py
+++ b/bodhi/server/metadata.py
@@ -74,7 +74,7 @@ def __init__(self, release, request, db, path):
         # Load from the cache if it exists
         self.cached_repodata = os.path.join(self.repo, '..', self.tag +
                                             '.repocache', 'repodata/')
-        if os.path.isdir(self.cached_repodata):
+        if os.path.isfile(os.path.join(self.cached_repodata, 'repomd.xml')):
             log.info('Loading cached updateinfo.xml')
             self._load_cached_updateinfo()
         else:


Test patch:
diff --git a/bodhi/tests/server/test_metadata.py b/bodhi/tests/server/test_metadata.py
--- a/bodhi/tests/server/test_metadata.py
+++ b/bodhi/tests/server/test_metadata.py
@@ -177,6 +177,50 @@ def get_notice(self, uinfo, title):
             if record.title == title:
                 return record
 
+    def test___init___checks_existence_if_repomd_xml(self):
+        """
+        The __init__() method has a test for finding cached repodata. It used to check for the
+        existence of a repodata folder, but this caused crashes sometimes because due to an unsolved
+        bug[0] this directory sometimes does not contain a repomd.xml file. This test ensures that
+        the existence of the repomd.xml file itself is tested to decide if it can load a cache.
+
+        [0] https://github.com/fedora-infra/bodhi/issues/887
+        """
+        update = self.db.query(Update).one()
+        # Pretend it's pushed to testing
+        update.status = UpdateStatus.testing
+        update.request = None
+        update.date_pushed = datetime.utcnow()
+        DevBuildsys.__tagged__[update.title] = ['f17-updates-testing']
+        # Generate the XML
+        md = ExtendedMetadata(update.release, update.request, self.db, self.temprepo)
+        # Insert the updateinfo.xml into the repository
+        md.insert_updateinfo()
+        md.cache_repodata()
+        updateinfo = self._verify_updateinfo(self.repodata)
+        # Change the notes on the update, but not the date_modified. Since this test deletes
+        # repomd.xml the cached notes should be ignored and we should see this 'x' in the notes.
+        # We can test for this to ensure that the cache was not used.
+        update.notes = u'x'
+        # Re-initialize our temporary repo
+        shutil.rmtree(self.temprepo)
+        os.mkdir(self.temprepo)
+        mkmetadatadir(join(self.temprepo, 'f17-updates-testing', 'i386'))
+        # Simulate the repomd.xml file missing. This should cause new updateinfo to be generated
+        # instead of it trying to load from the cache.
+        os.remove(
+            join(self.temprepo, '..', 'f17-updates-testing.repocache', 'repodata', 'repomd.xml'))
+
+        md = ExtendedMetadata(update.release, update.request, self.db, self.temprepo)
+
+        md.insert_updateinfo()
+        updateinfo = self._verify_updateinfo(self.repodata)
+        # Read and verify the updateinfo.xml.gz
+        uinfo = createrepo_c.UpdateInfo(updateinfo)
+        notice = self.get_notice(uinfo, update.title)
+        # Since 'x' made it into the xml, we know it didn't use the cache.
+        self.assertEquals(notice.description, u'x')  # not u'Useful details!'
+
     def test_extended_metadata(self):
         update = self.db.query(Update).one()