<PAGE>   1
                                                                    EXHIBIT 10.4


                       AMENDMENT NO. 3 TO VOTING AGREEMENT

        THIS AMENDMENT NO. 3 TO VOTING AGREEMENT ("Amendment") is made as of
April 28, 2000, by and among Discovery Partners International, Inc., a
California corporation formerly known as IRORI (the "Company"), Enterprise
Partners III, L.P., Enterprise Partners III Associates, L.P. (together,
"Enterprise"), Mayfield VIII, Mayfield Associates Fund II (together,
"Mayfield"), Crosspoint Venture Partners-1996, Crosspoint Venture Partners
LS-1997 (together, "Crosspoint"), Agilent Technologies, Inc. (formerly
Hewlett-Packard Company) ("Agilent"), Pacific Venture Group II, L.P.
("Pacific"), BayStar Capital, L.P. and BayStar International Ltd. (together
"BayStar"), Hayden J. Trubitt ("Trubitt") and Axys Pharmaceuticals, Inc.
("Axys"), with reference to the Voting Agreement, as amended and restated by
Amendment No. 2 to the Voting Agreement, dated April 7, 2000 (the "Voting
Agreement"), among the Company, Enterprise, Mayfield, Crosspoint, Agilent,
Pacific, Baystar and Trubitt. Enterprise, Mayfield, Crosspoint, Agilent,
Pacific, Baystar and Trubitt are collectively referred to herein as the
"Investors." RECITALS

        A. The Company, DPII Newco, LLC, a Delaware limited liability company
("Merger Sub"), Axys and Axys Advanced Technologies, Inc., a Delaware
corporation ("AAT"), have entered into that certain Agreement and Plan of
Merger, dated as of April 11, 2000 (the "Merger Agreement") pursuant to which
Merger Sub is being merged with and into AAT with the consequence that AAT will
become a wholly-owned subsidiary of the Company.

        B. Pursuant to the Company's Restated Articles of Incorporation, as
amended, the holders of the Company's Preferred Stock ("Preferred Stock") are
entitled to elect five (5) directors of the Company (the "Preferred Directors").

        C. This Amendment is entered into as a condition to and in consideration
of the Merger Agreement.

        NOW, THEREFORE, IT IS HEREBY AGREED THAT THE VOTING AGREEMENT IS AMENDED
AND RESTATED TO READ IN FULL AS FOLLOWS:

        1.     Agreement to Vote for Enterprise Nominee.

               a. During the term of this Voting Agreement, each of the
Investors agrees to vote all of its shares of the Company's Series A Preferred
Stock ("Series A Shares"), Series B Preferred Stock ("Series B Shares"), Series
C Preferred Stock ("Series C Shares"), Series D Preferred Stock ("Series D
Shares") and Series E Preferred Stock ("Series E Shares") now or hereafter owned
by it to elect one (1) nominee of Enterprise (the "Enterprise Nominee") as a
Preferred Director.

<PAGE>   2
               b. Prior to each election of directors of the Company, Enterprise
shall designate the Enterprise Nominee in writing to the Company. The Company
shall promptly notify each of the Investors of Enterprise's choice of such
nominee. Any vacancy occurring because of the death, resignation, removal or
disqualification of the Enterprise Nominee shall be filled according to this
Section 1.

        2.     Agreement to Vote for Mayfield Nominee.

               a. During the term of this Voting Agreement, each of the
Investors agrees to vote all of its Series A Shares, Series B Shares, Series C
Shares, Series D Shares and Series E Shares now or hereafter owned by it to
elect one (1) nominee of Mayfield (the "Mayfield Nominee") as a Preferred
Director.

               b. Prior to each election of directors of the Company, Mayfield
shall designate the Mayfield Nominee in writing to the Company. The Company
shall promptly notify each of the Investors of Mayfield's choice of such
nominee. Any vacancy occurring because of the death, resignation, removal or
disqualification of the Mayfield Nominee shall be filled according to this
Section 2.

        3.     Agreement to Vote for Crosspoint Nominee.

               a. During the term of this Voting Agreement, each of the
Investors agrees to vote all of its Series A Shares, Series B Shares, Series C
Shares, Series D Shares and Series E Shares now or hereafter owned by it to
elect one (1) nominee of Crosspoint (the "Crosspoint Nominee") as a Preferred
Director.

               b. Prior to each election of directors of the Company, Crosspoint
shall designate the Crosspoint Nominee in writing to the Company. The Company
shall promptly notify each of the Investors of Crosspoint's choice of such
nominee. Any vacancy occurring because of the death, resignation, removal or
disqualification of the Crosspoint Nominee shall be filled according to this
Section 3.

        4.     Agreement to Vote for Axys Nominee.

               a. During the term of this Voting Agreement, each of the
Investors agrees to vote all of its Series A Shares, Series B Shares, Series C
Shares, Series D Shares and Series E Shares now or hereafter owned by it to
elect one (1) nominee of Axys (the "Axys Nominee") as a Preferred Director.

               b. Prior to each election of directors of the Company, Axys shall
designate the Axys Nominee in writing to the Company. The Company shall promptly
notify each of the Investors of Axys' choice of such nominee. Any vacancy
occurring because of the death, resignation, removal or disqualification of the
Axys Nominee shall be filled according to this Section 3.


                                       2
<PAGE>   3
        5. Covenants of the Investors and the Company. The Company, Axys and the
Investors agree to use their reasonable best efforts to ensure that the rights
given to Axys and the Investors hereunder are effective and that Axys and the
Investors shall enjoy the benefits hereof. Such reasonable best efforts shall
include, without limitation, the use of the Company's, Axys' and the Investors'
reasonable best efforts to cause the nomination of the Enterprise Nominee, the
Mayfield Nominee, the Crosspoint Nominee and the Axys Nominee at each election
of the Board of Directors. Neither the Company, Axys nor any of the Investors
shall, by any voluntary action, avoid or seek to avoid the observance or
performance of any of the terms to be performed hereunder by the Company, Axys
or the Investors, respectively, but shall at all times in good faith assist in
the carrying out of all of the provisions of this Voting Agreement and shall use
their reasonable best efforts to protect the rights of the Investors and Axys
hereunder against impairment.

        6.     Successors and Assigns.

               a. The Investors' rights under this Agreement are not assignable
without the prior written consent of the Company and the holders of a majority
of the Preferred Stock then held by the Investors (the "Majority Investors"),
which consent may be withheld for any reason whatsoever. Axys' rights under this
Agreement are not assignable without the prior written consent of the Company,
which consent may be withheld for any reason whatsoever. Notwithstanding the
foregoing, Axys shall be allowed to make a bona fide pledge of its Company
Common Stock and Axys' rights under this Agreement may be assigned, on an
indivisible basis, to any person if both the following conditions apply: (i) the
assignment to such person is part of the enforcement against Axys' shares of
Company Common Stock of a security interest which Axys had, via a bona fide
pledge, granted to a lender, and (ii) the assignee complies with Section 6(b)
below.

               b. The burdens of this Voting Agreement shall be binding upon the
successors and assigns to whom the Investors transfer any of their respective
Series A Shares, Series B Shares, Series C Shares, Series D Shares and/or Series
E Shares. The Company shall not permit the transfer of any Series A Shares,
Series B Shares, Series C Shares, Series D Shares and/or Series E Shares on its
books or issue a new certificate representing any Series A Shares, Series B
Shares, Series C Shares, Series D Shares and/or Series E Shares unless and until
the person to whom such security is to be transferred shall have executed a
written agreement, reasonably satisfactory in form and substance to the Majority
Investors and the Company, pursuant to which such person becomes a party to this
Voting Agreement and agrees to be bound by all the burdens hereof as if such
person was an Investor hereunder.

               c. The provisions hereof shall inure to the benefit of, and be
binding upon, the successors and assigns of the Company. Without limitation on
any other rights of assignment, the parties acknowledge that if the Company
reincorporates in Delaware this Agreement shall be assigned to the new Delaware
corporation and the rights and obligations of the parties shall persist as if
all references herein to "the Company" were references to such Delaware
corporation.


                                       3
<PAGE>   4
        7. Legends. Each certificate representing any Series A Shares, Series B
Shares, Series C Shares, Series D Shares or Series E Shares held by the
Investors shall be endorsed by the Company with the following legend:

                THE SHARES EVIDENCED HEREBY ARE SUBJECT TO A VOTING AGREEMENT (A
                COPY OF WHICH MAY BE OBTAINED FROM THE ISSUER), AND BY ACCEPTING
                ANY INTEREST IN SUCH SHARES THE PERSON ACCEPTING SUCH INTEREST
                SHALL BE DEEMED TO AGREE TO AND SHALL BECOME BOUND BY ALL THE
                BURDENS OF SAID VOTING AGREEMENT.

        8. Termination. This Voting Agreement shall terminate upon the earlier
of (i) the consummation of the Company's initial public offering on a firm
underwriting basis, (ii) the closing of a consolidation or merger of the Company
with or into any other corporation or corporations, or a sale, conveyance or
disposition of all or substantially all of the assets of the Company or the
effectuation by the Company of a transaction or series of related transactions
in which more than 50% of the voting power of the Company is disposed of, (iii)
as to each Investor, that date when such Investor (a) holds less than 750,000
Series B Shares and/or Series A Shares, (b) holds less than 100,000 Series C
Shares or Series D Shares, and (c) holds less than 100,000 Series E Shares, or
(iv) as to Axys, that date when Axys holds less than 750,000 shares of the
Company's Common Stock.

        9. Amendments and Waivers. Any term hereof may be amended and the
observance of any term hereof may be waived (either generally or in a particular
instance and either retroactively or prospectively) only with the written
consent of the Company, the Majority Investors or their successors and permitted
assigns and Axys or its successors or permitted assigns. Any amendment or waiver
so effected shall be binding upon the Company, Axys and all of the Investors,
their successors and assigns.

        10. Severability. Whenever possible, each provision of this Voting
Agreement shall be interpreted in such manner as to be effective and valid under
applicable law, but if any provision of this Voting Agreement shall be held to
be prohibited by or invalid under applicable law, such provision shall be
ineffective only to the extent of such prohibition or invalidity, without
invalidating the remainder of such provision or the remaining provisions of this
Voting Agreement.

        11. Governing Law. This Agreement shall be governed by and construed
under the laws of the State of California, without regard to the conflict of
laws provisions thereof.

        12. Notices. All notices, requests and other communications to the
Company or any of the Investors hereunder shall be in writing (including
telecopy or similar electronic transmissions), shall refer specifically to this
Voting Agreement and shall be personally delivered or sent by telecopy or other
electronic facsimile transmission, overnight delivery with a nationally
recognized overnight delivery service or by registered or certified mail, return
receipt


                                       4
<PAGE>   5
requested, postage prepaid, in each case to the respective address specified
below (or such other address as may be specified in writing to the other parties
hereto):

                      a.     If to the Company, to:

                                    DISCOVERY PARTNERS INTERNATIONAL, INC.
                                    9640 Towne Center Drive
                                    San Diego, CA  92121
                                    Attention: Chief Executive Officer
                                    Fax No.:   (858) 455-8088

                      b.     If to Enterprise, to:

                                    ENTERPRISE PARTNERS
                                    7979 Ivanhoe Avenue, Suite 550
                                    La Jolla, California 92037
                                    Attention: Andrew E. Senyei,
                                               General Partner
                                    Fax No.:   (858) 454-2489

                      c.     If to Mayfield, to:

                                    MAYFIELD ASSOCIATES
                                    2800 Sand Hill Road
                                    Menlo Park, CA  94025
                                    Attention: A. Grant Heidrich,
                                               General Partner
                                    Fax No.:   (650) 854-5712

                      d.     If to Crosspoint, to:

                                    CROSSPOINT VENTURE PARTNERS
                                    18552 MacArthur Boulevard, Suite 400
                                    Irvine, CA  92715
                                    Attention: General Partner
                                    Fax No.:   (714) 852-9804

                      e.     If to Agilent, to:

                                    AGILENT TECHNOLOGIES, INC.
                                    3000 Hanover Street
                                    Palo Alto, CA  94304
                                    Attention: Director, Corporate Development
                                    Fax No.:   (650) 852-8432

                             With a copy to:

                                    AGILENT TECHNOLOGIES, INC.


                                       5
<PAGE>   6
                                    3000 Hanover Street
                                    Palo Alto, CA  94304
                                    Attention: General Counsel
                                    Fax No.:   (650) 852-8019

                      f.     If to Pacific, to:

                                    PACIFIC VENTURE GROUP
                                    15635 Alton Parkway, Suite 230
                                    Irvine, CA  92618
                                    Attention: Ralph C. Sabin
                                    Fax No.:   (949) 753-8932

                      g.     If to BayStar, to:

                                    BAYSTAR CAPITAL
                                    425 Market Street, 22nd Floor
                                    San Francisco, CA  94105
                                    Attention: Sherrill Lybrook, Principal
                                    Fax No.:   (415) 512-6472

                      h.     If to Trubitt, to:

                                    HAYDEN J. TRUBITT
                                    Brobeck, Phleger & Harrison LLP
                                    12390 El Camino Real
                                    San Diego, CA  92130
                                    Fax No.:   (858) 720-2555

                      i.     If to Axys, to:

                                    AXYS PHARMACEUTICALS, INC.
                                    180 Kimball Way
                                    South San Francisco, CA 94080
                                    Fax No.:   (650) 829-1067
                                    Attention: William J. Newell, Esq.

                             with a copy to:

                                    COOLEY GODWARD, LLP
                                    Five Palo Alto Square
                                    3000 El Camino Real
                                    Palo Alto, CA 94306-2155
                                    Fax No.:   (650) 849-7400
                                    Attention: Alan C. Mendelson, Esq.


                                       6
<PAGE>   7
Any notice or communication given in conformity with this Section 12 shall be
deemed to be effective when received by the addressee, if delivered by hand,
telecopy or electronic transmission, one (1) day after deposit with a nationally
recognized overnight delivery service and three (3) days after mailing, if
mailed.

        13. Equitable Remedies. The Company, the Investors and Axys acknowledge
and agree that the legal remedies available to the Investors and/or Axys in the
event any party violates the covenants and agreements made in this Voting
Agreement would be inadequate and that the Investors and/or Axys shall be
entitled, without posting any bond or other security, to temporary, preliminary,
and permanent injunctive relief, specific performance and other equitable
remedies in the event of such a violation, in addition to any other remedies
which the Investors and/or Axys may have at law or in equity.

        14. Counterparts. This Voting Agreement may be executed in any number
counterparts, each of which shall be deemed an original, but all of which
together shall constitute one and the same instrument.

        IN WITNESS WHEREOF, the parties hereto have executed this Amendment No.
3 to Voting Agreement as of the date first written above.

DISCOVERY PARTNERS, INC., a California corporation

By:  /s/ Riccardo Pigliucci
     -------------------------------------------
     Riccardo Pigliucci, Chief Executive Officer


ENTERPRISE PARTNERS III, L.P.

By:  Enterprise Management Partners III, L.P., Its General Partner

By:  /s/ Andrew Senyei
     -------------------------------------------
     General Partner


                                       7
<PAGE>   8
ENTERPRISE PARTNERS III ASSOCIATES, L.P.

By:  Enterprise Management Partners III, L.P., Its General Partner

By:  /s/ Andrew Senyei
     -------------------------------------------
     General Partner


MAYFIELD VIII, a California Limited Partnership

By:  MAYFIELD VIII MANAGEMENT, L.L.C., a Delaware Limited Liability
     Company, its General Partner

By:      /s/ A. Grant Heidrich, III
         --------------------------------------
Title:
         --------------------------------------


MAYFIELD ASSOCIATES FUND II, a California Limited Partnership

By:      /s/   A. Grant Heidrich, III
         --------------------------------------
         General Partner


CROSSPOINT VENTURE PARTNERS-1996

By:      /s/ Donald B. Milder
         --------------------------------------
         General Partner


CROSSPOINT VENTURE PARTNERS LS-1997

By:      /s/ Donald B. Milder
         --------------------------------------
         General Partner


                                       8
<PAGE>   9

AXYS PHARMACEUTICALS, INC., a Delaware corporation

By:    /s/ William J. Newell
       --------------------------------------
       William J. Newell, Senior Vice President


                                       9
<PAGE>   10
                     DISCOVERY PARTNERS INTERNATIONAL, INC.
                       AMENDMENT NO. 4 TO VOTING AGREEMENT


        This Amendment No. 4 to Voting Agreement (the "Amendment") is made as of
June 30, 2000 by and among Discovery Partners International, Inc., a California
corporation formerly known as IRORI (the "Company"), Enterprise Partners III,
L.P., Enterprise Partners III Associates, L.P. (together, "Enterprise"),
Mayfield VIII, Mayfield Associates Fund II (together, "Mayfield"), Crosspoint
Venture Partners-1996, Crosspoint Venture Partners LS-1997 (together,
"Crosspoint"), Agilent Technologies, Inc. (formerly Hewlett-Packard Company)
("Agilent"), Pacific Venture Group II, L.P. ("Pacific"), BayStar Capital, L.P.
and BayStar International Ltd. (together "BayStar"), Hayden J. Trubitt
("Trubitt"), Axys Pharmaceuticals, Inc. ("Axys") and PVG Associates II, L.P.
("PVG Associates), with reference to the Voting Agreement dated as of November
5, 1996 (together with Amendment No. 1 dated October 31, 1997, Amendment No. 2
dated April 7, 2000 and Amendment No. 3 dated April 28, 2000, the "Voting
Agreement"), among the Company, Enterprise, Mayfield, Crosspoint, Agilent,
Pacific, Baystar, Trubitt and Axys. Enterprise, Mayfield, Crosspoint, Agilent,
Pacific, Baystar, Trubitt and Axys are collectively referred to herein as the
"Existing Investors." Capitalized terms not otherwise defined herein shall have
the meanings set forth in the Voting Agreement.

                                    RECITALS

        A. Pacific desires to transfer 11,044 Series E Shares to PVG Associates;

        B. The Company and the Existing Investors desire that PVG Associates,
through this Amendment, enters into and becomes a part of this Voting Agreement,
having all rights and obligations of an Investor thereunder.

                                    AGREEMENT

        NOW, THEREFORE, in consideration of the mutual covenants set forth
herein, and for other good and valuable consideration the receipt of which is
hereby acknowledged, the parties agree as follows:

        1. Additional Party to the Voting Agreement. Pursuant to Section 9 of
the Voting Agreement, the Voting Agreement is hereby amended to include PVG
Associates as an Investor and PVG Associates shall be considered to be an
Investor for all purposes under the Voting Agreement.

        2. Amendments to Voting Agreement.

               (a) BayStar is confirmed to be an Investor at and after the date
of the Voting Agreement.

               (b) The parties confirm that the Voting Agreement has terminated
as to Trubitt due to Section 8(iii) of the Voting Agreement.


<PAGE>   11

               (c) For the purpose of the 100,000 Series E Shares threshold of
Section 8(iii)(c) of the Voting Agreement, all Series E Shares held by Pacific
and PVG Associates shall be deemed to be held by Pacific and shall also be
deemed to be held by PVG Associates.

        3. Effect of Amendment. Except as expressly modified by this Amendment,
the Voting Agreement shall remain unmodified and in full force and effect.

        4. Counterparts. This Amendment may be executed in two or more
counterparts, each of which shall be deemed an original, but all of which
together shall constitute one and the same instrument.

        5. Severability. In the event one or more of the provisions of this
Amendment should, for any reason, be held to be invalid, illegal or
unenforceable in any respect, such invalidity, illegality or unenforceability
shall not affect any other provisions of this Amendment, and this Amendment
shall be construed and interpreted in such manner as to be effective and valid
under applicable law.



                [REMAINDER OF THIS PAGE INTENTIONALLY LEFT BLANK]



                                       2
<PAGE>   12
        IN WITNESS WHEREOF, the parties hereto have executed this Amendment as
of the day and year first above written.

                                Discovery Partners International, Inc., a
                                California corporation (formerly known as IRORI)


                                By: /s/ Riccardo Pigliucci
                                    --------------------------------------------
                                    Riccardo Pigliucci, Chief Executive Officer



                                INVESTORS:


                                ENTERPRISE PARTNERS III, L.P.


                                By:     Enterprise Management Partners III,
                                        L.P., Its General Partner


                                        By: /s/ Andrew Senyei
                                            ------------------------------------
                                            General Partner, Andrew E. Senyei,
                                              M.D.


                                ENTERPRISE PARTNERS III ASSOCIATES, L.P.


                                By:     Enterprise Management Partners III,
                                        L.P., Its General Partner


                                        By: /s/ Andrew Senyei
                                            ------------------------------------
                                            General Partner, Andrew E. Senyei,
                                              M.D.


                                PACIFIC VENTURE GROUP II, L.P.


                                By: /s/ Ralph C. Sabin
                                   ---------------------------------------------
                                Its: Managing Director
                                    --------------------------------------------



             [SIGNATURE PAGE TO AMENDMENT NO. 4 TO VOTING AGREEMENT]
<PAGE>   13

                                MAYFIELD VIII, a California Limited Partnership


                                By:      Mayfield VIII Management, L.L.C. a
                                         Delaware limited liability company, its
                                          General Partner


                                         By: /s/ A. Grant Heidrich
                                             -----------------------------------
                                             General Partner


                                MAYFIELD ASSOCIATES FUND II, a California
                                Limited Partnership


                                By:      /s/ A. Grant Heidrich
                                         ---------------------------------------
                                          General Partner


                                CROSSPOINT VENTURE PARTNERS-1996


                                By: /s/ Don Milder
                                    --------------------------------------------
                                       Don Milder, General Partner


                                CROSSPOINT VENTURE PARTNERS LS-1997

                                By: /s/ Don Milder
                                    --------------------------------------------
                                       Don Milder, General Partner


                                AGILENT TECHNOLOGIES, INC., f/k/a
                                Hewlett-Packard Company


                                By:
                                    --------------------------------------------

                                Its:
                                    --------------------------------------------



                                BRISTOL-MYERS SQUIBB COMPANY


                                By:
                                    --------------------------------------------


                                Its:
                                    --------------------------------------------



             [SIGNATURE PAGE TO AMENDMENT NO. 4 TO VOTING AGREEMENT]
<PAGE>   14

                              RHONE-POULENC RORER INTERNATIONAL HOLDINGS, INC.


                              By:
                                    --------------------------------------------

                              Its:
                                    --------------------------------------------


                              BAYSTAR CAPITAL, LLP, a Delaware Limited
                              Partnership

                              By:    BayStar Capital Management, LLC,
                                     Its General Partner

                              By: /s/ Steve Lamar
                                  ----------------------------------------------
                                      Steve Lamar, Vice President


                              BAYSTAR INTERNATIONAL LTD, a British Virgin
                              Islands Corporation


                              By:    BayStar International Management, LLC,
                                     Its General Partner

                              By: /s/ Steve Lamar
                                  ----------------------------------------------
                                      Steve Lamar, Vice President


                              --------------------------------------------------
                                              Hayden J. Trubitt


                              AXYS PHARMACEUTICALS, INC. a Delaware Corporation


                              By: /s/ William J. Newell
                                  ----------------------------------------------
                                      William J. Newell, Senior Vice President



             [SIGNATURE PAGE TO AMENDMENT NO. 4 TO VOTING AGREEMENT]
<PAGE>   15

                              PVG ASSOCIATES II, L.P.


                              By:     PVG Equity Partners II, L.L.C.
                                      its General Partner


                                      By: /s/ Ralph C. Sabin
                                         ---------------------------------------
                                         Ralph C. Sabin, Managing Director


                                      16830 Ventura Boulevard
                                      Suite 244
                                      Encino, California 91436



             [SIGNATURE PAGE TO AMENDMENT NO. 4 TO VOTING AGREEMENT]