CI Build for Bitcoin - Some Basic Questions about Gitian and other stuff

CI Build for Bitcoin - Some Basic Questions about Gitian and other stuff
2015-09-23 17:18 ` Mark Friedenbach
  
From: Roy Osherove @ 2015-09-23 17:13 UTC (permalink / raw)
  To: bitcoin-dev

[-- Attachment #1: Type: text/plain, Size: 1413 bytes --]

Hi Folks.
I'm trying my hand at creating a reproducible build of my own for bitcoin
and bitcoin-XT, using TeamCity.
I believe it is the best way to learn something: To try to build it
yourself.
Here is what I think I know so far, and I would love corrections, plus
questions:

   1. Bitcoin is built continuously on travis-CI at
   https://travis-ci.org/bitcoin/bitcoin/
   2.  there are many flavors that are built, but I'm not sure if all of
   them are actually used/necessary. are they all needed, or just to "just in
   case"?
   3.  There is a gitian build file for bitcoin, but is anyone actually
   using it? are the bin files on bitcoin.org taken from that? or the
   travis ci builds? or some other place?
   4. Are there any things that people would love to have in the build that
   do not exist there today? perhaps I can help with that?

Here is what I have now: http://btcdev.osherove.com:8111/
It does not do the matrix build yet, but it's coming. I'm just wondering if
all the platforms need to be supported,and if gitian is truly required to
be used, or used in parallel, or at all..

Feedback appreciated.

-- 
Thanks,

Roy Osherove

   - *@RoyOsherove* <https://twitter.com/RoyOsherove>
   - Read my new book *Notes to a Software Team Leader
<http://TeamLeadSkills.com>*
   - Or *my new course* about Beautiful Builds <http://courses.osherove.com>
and Continuous Delivery
   - +1-201-256-5575

[-- Attachment #2: Type: text/html, Size: 2005 bytes --]

 permalink raw reply	[|nested]
2015-09-23 17:13 [bitcoin-dev] CI Build for Bitcoin - Some Basic Questions about Gitian and other stuff Roy Osherove

  2015-09-23 17:31   ` Roy Osherove
  
From: Mark Friedenbach @ 2015-09-23 17:18 UTC (permalink / raw)
  To: Roy Osherove;  Bitcoin Dev

[-- Attachment #1: Type: text/plain, Size: 2115 bytes --]

The builds made by Travis are for the purpose of making sure that the
source code compiles and tests run successfully on all supported platforms.
The binaries are not used anywhere else because Travis is not a trusted
platform.

The binaries on bitcoin.org are built using the gitian process and signed
by a quorum of developers.

On Wed, Sep 23, 2015 at 10:13 AM, Roy Osherove via bitcoin-dev <
bitcoin-dev@lists•linuxfoundation.org> wrote:

> Hi Folks.
> I'm trying my hand at creating a reproducible build of my own for bitcoin
> and bitcoin-XT, using TeamCity.
> I believe it is the best way to learn something: To try to build it
> yourself.
> Here is what I think I know so far, and I would love corrections, plus
> questions:
>
>    1. Bitcoin is built continuously on travis-CI at
>    https://travis-ci.org/bitcoin/bitcoin/
>    2.  there are many flavors that are built, but I'm not sure if all of
>    them are actually used/necessary. are they all needed, or just to "just in
>    case"?
>    3.  There is a gitian build file for bitcoin, but is anyone actually
>    using it? are the bin files on bitcoin.org taken from that? or the
>    travis ci builds? or some other place?
>    4. Are there any things that people would love to have in the build
>    that do not exist there today? perhaps I can help with that?
>
> Here is what I have now: http://btcdev.osherove.com:8111/
> It does not do the matrix build yet, but it's coming. I'm just wondering
> if all the platforms need to be supported,and if gitian is truly required
> to be used, or used in parallel, or at all..
>
> Feedback appreciated.
>
> --
> Thanks,
>
> Roy Osherove
>
>    - *@RoyOsherove* <https://twitter.com/RoyOsherove>
>    - Read my new book *Notes to a Software Team Leader
> <http://TeamLeadSkills.com>*
>    - Or *my new course* about Beautiful Builds
> <http://courses.osherove.com> and Continuous Delivery
>    - +1-201-256-5575
>
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>

[-- Attachment #2: Type: text/html, Size: 3338 bytes --]

 permalink raw reply	[|nested]
2015-09-23 17:18 ` Mark Friedenbach

  2015-09-23 18:10     ` Mark Friedenbach
  
From: Roy Osherove @ 2015-09-23 17:31 UTC (permalink / raw)
  To: Mark Friedenbach;  Bitcoin Dev

[-- Attachment #1: Type: text/plain, Size: 2819 bytes --]

Thanks Mark.
Is there a public server where the gitian builds can be viewed?
Is there a public server that shows the quorum verifications or that shows
how to join in on the verification if such as thing is helpful?

On Wed, Sep 23, 2015 at 10:18 AM, Mark Friedenbach <mark@friedenbach•org>
wrote:

> The builds made by Travis are for the purpose of making sure that the
> source code compiles and tests run successfully on all supported platforms.
> The binaries are not used anywhere else because Travis is not a trusted
> platform.
>
> The binaries on bitcoin.org are built using the gitian process and signed
> by a quorum of developers.
>
> On Wed, Sep 23, 2015 at 10:13 AM, Roy Osherove via bitcoin-dev <
> bitcoin-dev@lists•linuxfoundation.org> wrote:
>
>> Hi Folks.
>> I'm trying my hand at creating a reproducible build of my own for bitcoin
>> and bitcoin-XT, using TeamCity.
>> I believe it is the best way to learn something: To try to build it
>> yourself.
>> Here is what I think I know so far, and I would love corrections, plus
>> questions:
>>
>>    1. Bitcoin is built continuously on travis-CI at
>>    https://travis-ci.org/bitcoin/bitcoin/
>>    2.  there are many flavors that are built, but I'm not sure if all of
>>    them are actually used/necessary. are they all needed, or just to "just in
>>    case"?
>>    3.  There is a gitian build file for bitcoin, but is anyone actually
>>    using it? are the bin files on bitcoin.org taken from that? or the
>>    travis ci builds? or some other place?
>>    4. Are there any things that people would love to have in the build
>>    that do not exist there today? perhaps I can help with that?
>>
>> Here is what I have now: http://btcdev.osherove.com:8111/
>> It does not do the matrix build yet, but it's coming. I'm just wondering
>> if all the platforms need to be supported,and if gitian is truly required
>> to be used, or used in parallel, or at all..
>>
>> Feedback appreciated.
>>
>> --
>> Thanks,
>>
>> Roy Osherove
>>
>>    - *@RoyOsherove* <https://twitter.com/RoyOsherove>
>>    - Read my new book *Notes to a Software Team Leader
>> <http://TeamLeadSkills.com>*
>>    - Or *my new course* about Beautiful Builds
>> <http://courses.osherove.com> and Continuous Delivery
>>    - +1-201-256-5575
>>
>>
>>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists•linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>>
>


-- 
Thanks,

Roy Osherove

   - *@RoyOsherove* <https://twitter.com/RoyOsherove>
   - Read my new book *Notes to a Software Team Leader
<http://TeamLeadSkills.com>*
   - Or *my new course* about Beautiful Builds <http://courses.osherove.com>
and Continuous Delivery
   - +1-201-256-5575
    - Timezone: Eastern Standard Time (New York)

[-- Attachment #2: Type: text/html, Size: 4701 bytes --]

 permalink raw reply	[|nested]
2015-09-23 17:31   ` Roy Osherove

  2015-09-23 20:54       ` Roy Osherove
  
From: Mark Friedenbach @ 2015-09-23 18:10 UTC (permalink / raw)
  To: Roy Osherove;  Bitcoin Dev

[-- Attachment #1: Type: text/plain, Size: 3298 bytes --]

Well the gitian builds are made available on bitcoin.org. If you mean a
build server where gitian builds are automatically done and made available,
well that rather defeats the point of gitian.

The quorum signatures are accumulated here:
https://github.com/bitcoin/gitian.sigs (it's a manual process).

On Wed, Sep 23, 2015 at 10:31 AM, Roy Osherove <roy@osherove•com> wrote:

> Thanks Mark.
> Is there a public server where the gitian builds can be viewed?
> Is there a public server that shows the quorum verifications or that shows
> how to join in on the verification if such as thing is helpful?
>
> On Wed, Sep 23, 2015 at 10:18 AM, Mark Friedenbach <mark@friedenbach•org>
> wrote:
>
>> The builds made by Travis are for the purpose of making sure that the
>> source code compiles and tests run successfully on all supported platforms.
>> The binaries are not used anywhere else because Travis is not a trusted
>> platform.
>>
>> The binaries on bitcoin.org are built using the gitian process and
>> signed by a quorum of developers.
>>
>> On Wed, Sep 23, 2015 at 10:13 AM, Roy Osherove via bitcoin-dev <
>> bitcoin-dev@lists•linuxfoundation.org> wrote:
>>
>>> Hi Folks.
>>> I'm trying my hand at creating a reproducible build of my own for
>>> bitcoin and bitcoin-XT, using TeamCity.
>>> I believe it is the best way to learn something: To try to build it
>>> yourself.
>>> Here is what I think I know so far, and I would love corrections, plus
>>> questions:
>>>
>>>    1. Bitcoin is built continuously on travis-CI at
>>>    https://travis-ci.org/bitcoin/bitcoin/
>>>    2.  there are many flavors that are built, but I'm not sure if all
>>>    of them are actually used/necessary. are they all needed, or just to "just
>>>    in case"?
>>>    3.  There is a gitian build file for bitcoin, but is anyone actually
>>>    using it? are the bin files on bitcoin.org taken from that? or the
>>>    travis ci builds? or some other place?
>>>    4. Are there any things that people would love to have in the build
>>>    that do not exist there today? perhaps I can help with that?
>>>
>>> Here is what I have now: http://btcdev.osherove.com:8111/
>>> It does not do the matrix build yet, but it's coming. I'm just wondering
>>> if all the platforms need to be supported,and if gitian is truly required
>>> to be used, or used in parallel, or at all..
>>>
>>> Feedback appreciated.
>>>
>>> --
>>> Thanks,
>>>
>>> Roy Osherove
>>>
>>>    - *@RoyOsherove* <https://twitter.com/RoyOsherove>
>>>    - Read my new book *Notes to a Software Team Leader
>>> <http://TeamLeadSkills.com>*
>>>    - Or *my new course* about Beautiful Builds
>>> <http://courses.osherove.com> and Continuous Delivery
>>>    - +1-201-256-5575
>>>
>>>
>>>
>>> _______________________________________________
>>> bitcoin-dev mailing list
>>> bitcoin-dev@lists•linuxfoundation.org
>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>
>>>
>>
>
>
> --
> Thanks,
>
> Roy Osherove
>
>    - *@RoyOsherove* <https://twitter.com/RoyOsherove>
>    - Read my new book *Notes to a Software Team Leader
> <http://TeamLeadSkills.com>*
>    - Or *my new course* about Beautiful Builds
> <http://courses.osherove.com> and Continuous Delivery
>    - +1-201-256-5575
>     - Timezone: Eastern Standard Time (New York)
>
>
>

[-- Attachment #2: Type: text/html, Size: 5623 bytes --]

 permalink raw reply	[|nested]
2015-09-23 18:10     ` Mark Friedenbach

  2015-09-24  9:18         ` Jonas Schnelli
  
From: Roy Osherove @ 2015-09-23 20:54 UTC (permalink / raw)
  To: Mark Friedenbach;  Bitcoin Dev

[-- Attachment #1: Type: text/plain, Size: 3887 bytes --]

So who physically manually uploads the gitian build to bitcoin.org?

On Wed, Sep 23, 2015 at 11:10 AM, Mark Friedenbach <mark@friedenbach•org>
wrote:

> Well the gitian builds are made available on bitcoin.org. If you mean a
> build server where gitian builds are automatically done and made available,
> well that rather defeats the point of gitian.
>
> The quorum signatures are accumulated here:
> https://github.com/bitcoin/gitian.sigs (it's a manual process).
>
> On Wed, Sep 23, 2015 at 10:31 AM, Roy Osherove <roy@osherove•com> wrote:
>
>> Thanks Mark.
>> Is there a public server where the gitian builds can be viewed?
>> Is there a public server that shows the quorum verifications or that
>> shows how to join in on the verification if such as thing is helpful?
>>
>> On Wed, Sep 23, 2015 at 10:18 AM, Mark Friedenbach <mark@friedenbach•org>
>> wrote:
>>
>>> The builds made by Travis are for the purpose of making sure that the
>>> source code compiles and tests run successfully on all supported platforms.
>>> The binaries are not used anywhere else because Travis is not a trusted
>>> platform.
>>>
>>> The binaries on bitcoin.org are built using the gitian process and
>>> signed by a quorum of developers.
>>>
>>> On Wed, Sep 23, 2015 at 10:13 AM, Roy Osherove via bitcoin-dev <
>>> bitcoin-dev@lists•linuxfoundation.org> wrote:
>>>
>>>> Hi Folks.
>>>> I'm trying my hand at creating a reproducible build of my own for
>>>> bitcoin and bitcoin-XT, using TeamCity.
>>>> I believe it is the best way to learn something: To try to build it
>>>> yourself.
>>>> Here is what I think I know so far, and I would love corrections, plus
>>>> questions:
>>>>
>>>>    1. Bitcoin is built continuously on travis-CI at
>>>>    https://travis-ci.org/bitcoin/bitcoin/
>>>>    2.  there are many flavors that are built, but I'm not sure if all
>>>>    of them are actually used/necessary. are they all needed, or just to "just
>>>>    in case"?
>>>>    3.  There is a gitian build file for bitcoin, but is anyone
>>>>    actually using it? are the bin files on bitcoin.org taken from
>>>>    that? or the travis ci builds? or some other place?
>>>>    4. Are there any things that people would love to have in the build
>>>>    that do not exist there today? perhaps I can help with that?
>>>>
>>>> Here is what I have now: http://btcdev.osherove.com:8111/
>>>> It does not do the matrix build yet, but it's coming. I'm just
>>>> wondering if all the platforms need to be supported,and if gitian is truly
>>>> required to be used, or used in parallel, or at all..
>>>>
>>>> Feedback appreciated.
>>>>
>>>> --
>>>> Thanks,
>>>>
>>>> Roy Osherove
>>>>
>>>>    - *@RoyOsherove* <https://twitter.com/RoyOsherove>
>>>>    - Read my new book *Notes to a Software Team Leader
>>>> <http://TeamLeadSkills.com>*
>>>>    - Or *my new course* about Beautiful Builds
>>>> <http://courses.osherove.com> and Continuous Delivery
>>>>    - +1-201-256-5575
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> bitcoin-dev mailing list
>>>> bitcoin-dev@lists•linuxfoundation.org
>>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>>
>>>>
>>>
>>
>>
>> --
>> Thanks,
>>
>> Roy Osherove
>>
>>    - *@RoyOsherove* <https://twitter.com/RoyOsherove>
>>    - Read my new book *Notes to a Software Team Leader
>> <http://TeamLeadSkills.com>*
>>    - Or *my new course* about Beautiful Builds
>> <http://courses.osherove.com> and Continuous Delivery
>>    - +1-201-256-5575
>>     - Timezone: Eastern Standard Time (New York)
>>
>>
>>
>


-- 
Thanks,

Roy Osherove

   - *@RoyOsherove* <https://twitter.com/RoyOsherove>
   - Read my new book *Notes to a Software Team Leader
<http://TeamLeadSkills.com>*
   - Or *my new course* about Beautiful Builds <http://courses.osherove.com>
and Continuous Delivery
   - +1-201-256-5575
    - Timezone: Eastern Standard Time (New York)

[-- Attachment #2: Type: text/html, Size: 6797 bytes --]

 permalink raw reply	[|nested]
2015-09-23 20:54       ` Roy Osherove

  
From: Jonas Schnelli @ 2015-09-24  9:18 UTC (permalink / raw)
  To: bitcoin-dev

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi Roy

> So who physically manually uploads the gitian build to bitcoin.org 
> <http://bitcoin.org>?

It's done by the core devs (very likely the project maintainer).
I think it doesn't matter who did the upload to bitcoin.org.

What really would matter is – if users deciding to run a pre-compiled
version of bitcoin-core – that they verify the binary against the
available gitian sigs.

Signatur repository:
https://github.com/bitcoin/gitian.sigs

PGP Pubkeys (mostly also available over gpg key servers):
https://github.com/bitcoin/bitcoin/tree/master/contrib/gitian-downloader

Instruction available here:
https://www.reddit.com/r/Bitcoin/wiki/verifying_bitcoin_core


/jonas
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJWA7/YAAoJECnUvLZBb1PspoEQAKDs672Vn/EFEn02oYbo41Xi
tiHLpZMePiL1GV+XAQrqA91x5Q4RY2oJBmBzq2Bnllr+fm5zodcTP80lDyYeizb7
+2MPuoF8ICMYxUDJ3tz163Y4hHforFyMMHgr/NPXQcsMEEnEuAQIMxRFMqexhkn9
W/YVT2ow/5illYmZ9EAoSreaD+1ShVTxkZY2ltY79ZATTcVU85mEXau9Jv6qq4sN
jHCWAJYrsFQay4t42Pp4ciclz3A7W//OoFhwxUR2sNcDZ6NkOSnYOZSwMPM4uBAL
z1OpSWVOSQZeGXFOzEF07oyLaCVKm6ygND4IT7eFcvlrgXMIYE0bvzfqPymBArRJ
ZSKR0bmTkUsuw3NCcV23HF8yW+G/Y/SKPRl5n1AmmXppzklSSojDzDT3h2xKGNTd
DYoIQhwsNcm7sLlmHZa+VZ7peeCzdQ9z+OTG1ZDwDsFRBDY9kTbP2pD1tHEkEvxR
z2DJg8iP9V4ZUUFEuAdOOCInl1v+0RJu8DxWa6BZmLYJ/SVuKqSEZ0HYBqPOJOUO
4ct+NLyY7fNQurq0VzIjpZN9/L+SEePSIP4bFwOiGBJosUEHyS3VZ7JZZMOs5jxx
1GDI964Pi54Z/XPzV2+X1GjWLeReJ6WUznX7zv/LNER2yI6XQBw2w5/rmomI+ifl
vk5pfinRCIP3t6q+K0ge
=vP2n
-----END PGP SIGNATURE-----


 permalink raw reply	[|nested]