<PAGE>   1
                                                                     EXHIBIT 3.1

                          SECOND AMENDED AND RESTATED

                          CERTIFICATE OF INCORPORATION

                                       OF

                           COSINE COMMUNICATIONS, INC.



        CoSine Communications, Inc., a corporation organized and existing under
the laws of the State of Delaware, hereby certifies as follows:

         A. The name of the corporation is CoSine Communications, Inc. The
corporation was originally incorporated under the same name and the original
Certificate of Incorporation of the corporation was filed with the Secretary of
State of the State of Delaware on May 16, 2000.

         B. This Second Amended and Restated Certificate of Incorporation
restates and integrates and further amends the provisions of the corporation's
Certificate of Incorporation as heretofore amended. The amendments and
restatement herein set forth have been duly approved by the corporation's Board
of Directors and stockholders in accordance with Sections 242 and 245 of the
General Corporation Law of the State of Delaware, with the approval of the
corporation's stockholders having been given by written consent in accordance
with Section 228 of the General Corporation Law of the State of Delaware.

        C. The Certificate of Incorporation is hereby restated and further
amended to read in its entirety as follows:


                                   ARTICLE I.

 The name of the corporation is CoSine Communications, Inc. (the "Corporation").



<PAGE>   2



                                   ARTICLE II.

        The address of the Corporation's registered office in the State of
Delaware is 1013 Centre Road, City of Wilmington, County of New Castle, Delaware
19801. The name of its registered agent at such address is the Corporation
Service Company.


                                  ARTICLE III.

        The nature of the business or purposes to be conducted or promoted by
the Corporation is to engage in any lawful act or activity for which
corporations may be organized under the General Corporation Law of Delaware.


                                   ARTICLE IV.

        The total number of shares of all classes of stock which the Corporation
shall have authority to issue is 303,000,000, of which (i) 300,000,000 shares,
par value $0.0001 per share, are to be of a class designated Common Stock
("Common Stock") and (ii) 3,000,000 shares, par value $0.0001 per share, are to
be of a class designated Preferred Stock ("Preferred Stock"). The Preferred
Stock may be issued from time to time in one or more series. The Board of
Directors is hereby authorized, subject to any limitations prescribed by law, to
fix by resolution or resolutions the designations, powers, preferences and
rights, and the qualifications, limitations or restrictions thereof, of each
such series of Preferred Stock, including without limitation authority to fix by
resolution or resolutions, the dividend rights, dividend rate, conversion
rights, voting rights, rights and terms of redemption (including sinking fund
provisions), redemption price or prices, and liquidation preferences of any
wholly unissued series of Preferred Stock, and the number of shares constituting
any such series and the designation thereof, or any of the foregoing.

        The Board of Directors is further authorized to increase (but not above
the total number of authorized shares of the class) or decrease (but not below
the number of shares of any such series then outstanding) the number of shares
of any series, the number of which was fixed by it, subsequent to the issue of
shares of such series then outstanding, subject to the powers, preferences and
rights, and the qualifications, limitations and restrictions thereof stated in
the resolution of the Board of Directors originally fixing the number of shares
of such series. If the number of shares of any series is so decreased, then the
shares constituting such decrease shall resume the status which they had prior
to the adoption of the resolution originally fixing the number of shares of such
series.


                                   ARTICLE V.

        The Corporation is to have perpetual existence.


                                      -2-
<PAGE>   3

                                   ARTICLE VI.

        Elections of directors need not be by written ballot unless the Bylaws
of the Corporation shall so provide.


                                  ARTICLE VII.

        1. The management of the business and the conduct of the affairs of the
Corporation shall be vested in its Board of Directors. The number of directors
which constitute the whole Board of Directors of the Corporation shall be fixed
by, or determined in the manner provided in, the Bylaws of the Corporation.

        2. Subject to the rights of the holders of any series of Preferred Stock
to elect additional directors under specified circumstances, the Board of
Directors shall be divided into three classes, as nearly equal in size as is
practicable, designated as Class I, Class II and Class III, respectively.
Directors shall be assigned to each class in accordance with a resolution or
resolutions adopted by the Board of Directors. At the first annual meeting of
stockholders following the date of the closing of the Corporation's initial
public offering pursuant to an effective registration statement under the
Securities Act of 1933, as amended, covering the offer and sale of Common Stock
to the public (the "Initial Public Offering"), the term of office of the Class I
directors shall expire and Class I directors shall be elected for a full term of
three years. At the second annual meeting of stockholders following the date of
the Initial Public Offering, the term of office of the Class II directors shall
expire and Class II directors shall be elected for a full term of three years.
At the third annual meeting of stockholders following the date of the Initial
Public Offering, the term of office of the Class III directors shall expire and
Class III directors shall be elected for a full term of three years. At each
succeeding annual meeting of stockholders, directors shall be elected for a full
term of three years to succeed the directors of the class whose terms expire at
such annual meeting.

        3. Notwithstanding the foregoing provisions of this Article VII, each
director shall serve until such director's successor is duly elected and
qualified or until such director's earlier death, resignation or removal. If the
number of directors is hereafter changed, any newly created directorships or
decrease in directorships shall be so apportioned among the classes as to make
all classes as nearly equal in number as is practicable, provided that no
decrease in the number of directors constituting the Board of Directors shall
shorten the term of any incumbent director.

        4. Subject to the rights of the holders of any series of Preferred
Stock, any vacancies on the Board of Directors resulting from death,
resignation, disqualification, removal, or other causes shall be filled by
either (i) the affirmative vote of the holders of a majority of the voting power
of the then-outstanding shares of voting stock of the corporation entitled to
vote generally in the election of directors ("Voting Stock") voting together as
a single class; or (ii) by the affirmative vote of a majority of the remaining
directors then in office, even though less than a quorum of the Board of
Directors. Newly created directorships resulting from any increase in the number
of directors shall, unless the Board of Directors determines by resolution that
any such newly created directorship shall be filled by the stockholders, be
filled only by the affirmative vote of the directors then in office, even though
less than a quorum of the Board of Directors. Any director elected in accordance
with

                                      -3-

<PAGE>   4

the preceding two sentences shall hold office for the remainder of the full term
of the class of directors in which the new directorship was created or the
vacancy occurred.

        5. The affirmative vote of sixty-six and two-thirds percent (66-2/3%) of
the voting power of the then outstanding shares of Voting Stock, voting together
as a single class, shall be required for the adoption, amendment or repeal of
the following sections or Articles of the corporation's Bylaws by the
stockholders of this corporation: 2.2 (Annual Meeting), 2.3 (Special Meeting),
2.5 (Advance Notice of Stockholder Nominees and Stockholder Business), 3.2
(Number of Directors) and Article VI (Indemnification).

        6. No action shall be taken by the stockholders of the corporation
except at an annual or special meeting of the stockholders called in accordance
with the Bylaws, and no action shall be taken by the stockholders by written
consent.

        7. Subject to the rights of any series of Preferred Stock, any director,
or the entire Board of Directors, may be removed from office at any time (i)
with cause by the affirmative vote of the holders of at least a majority of the
voting power of all of the then-outstanding shares of the Voting Stock, voting
together as a single class; or (ii) without cause by the affirmative vote of the
holders of at least sixty-six and two-thirds percent (66-2/3%) of the voting
power of all of the then-outstanding shares of the Voting Stock.


                                 ARTICLE VIII.

        In furtherance and not in limitation of the powers conferred by statute,
the Board of Directors is expressly authorized to adopt, alter, amend or repeal
the Bylaws of the Corporation.


                                  ARTICLE IX.

        1. To the fullest extent permitted by the Delaware General Corporation
Law as the same exists or as may hereafter be amended, no director of the
Corporation shall be personally liable to the Corporation or its stockholders
for monetary damages for breach of fiduciary duty as a director.

        2. The Corporation may indemnify to the fullest extent permitted by law
any person made or threatened to be made a party to any threatened, pending or
completed action, suit or proceeding, whether criminal, civil, administrative or
investigative, by reason of the fact that he, his testator or intestate is or
was a director, officer, employee or agent of the Corporation or any predecessor
of the Corporation or serves or served at any other enterprise as a director,
officer, employee or agent at the request of the Corporation or any predecessor
to the Corporation.

        3. Neither any amendment nor repeal of this Article XI, nor the adoption
of any provision of this Certificate of Incorporation inconsistent with this
Article XI, shall eliminate or reduce the effect of this Article XI, in respect
of any matter occurring, or any action or proceeding accruing or arising or
that, but for this Article XI, would accrue or arise, prior to such amendment,
repeal or adoption of an inconsistent provision.


                                      -4-

<PAGE>   5

                                   ARTICLE X.

        Advance notice of stockholder nominations for the election of directors
of the Corporation and of business to be brought by stockholders before any
meeting of stockholders of the Corporation shall be given in the manner and to
the extent provided in the Bylaws of the Corporation. Business transacted at
special meetings of stockholders shall be confined to the purpose or purposes
stated in the notice of meeting.


                                   ARTICLE XI.

        Notwithstanding any other provisions of this Certificate of
Incorporation or any provision of law that might otherwise permit a lesser vote
or no vote, but in addition to any affirmative vote of the holders of any
particular class or series of Voting Stock required by law, this Certificate of
Incorporation or any Preferred Stock Designation, the affirmative vote of the
holders of at least sixty-six and two-thirds percent (66-2/3%) of the voting
power of all of the then-outstanding shares of Voting Stock, voting together as
a single class, shall be required to alter, amend or repeal Article VI, Article
VII, Article VIII, Article IX, Article X or this Article XI.


                                  ARTICLE XII.

        The Corporation reserves the right to amend, alter, change or repeal any
provision contained in this Certificate of Incorporation, in the manner now or
hereafter prescribed by statute (subject to compliance with the requirements of
Article X), and all rights conferred herein are granted subject to this
reservation.


                                  ARTICLE XIII.

        Meetings of stockholders may be held within or without the State of
Delaware, as the Bylaws may provide. The books of the Corporation may be kept
(subject to any provision contained in the laws of the State of Delaware)
outside of the State of Delaware at such place or places as may be designated
from time to time by the Board of Directors or in the Bylaws of the Corporation.


        D. The foregoing Second Amended and Restated Certificate of
Incorporation of CoSine Communications, Inc. is the act and deed of this
corporation, and the statements therein are true.

                                      -5-

<PAGE>   6

        IN WITNESS WHEREOF, the Board of Directors of the Company has caused
this Second Amended and Restated Certificate of Incorporation to be signed by
Dean Hamilton, its President and Chief Executive Officer.



Dated:  September 29, 2000.             COSINE COMMUNICATIONS, INC.



                                        By: /s/ DEAN HAMILTON
                                           -------------------------------------
                                           Dean Hamilton
                                           President and Chief Executive Officer


                                      -6-