<PAGE>

                                                                   EXHIBIT 10.23
                       AMENDMENT TO ESCONDIDO BRANCH LEASE

                            FIFTH AMENDMENT TO LEASE

This Fifth Amendment to Lease ("this Amendment") is entered effective June 6,
2001 by and between Camp Crook Properties, LLC a California limited liability
company ("Landlord") and Palomar Community Bank, a California Banking
corporation ("Tenant").

1.     Recitals

       A.   On April 15, 1987, Palomar Plaza Associates Limited, a California
            general partnership, and Palomar Savings and Loan Association, a
            California corporation ("Tenant") entered into a written lease ("the
            Lease") for the real property located at 355 Grand Avenue,
            Escondido, California, described as the first floor of the building
            ("the Palomar Bank Building") constituting 7,000 square feet ("the
            Premises").

       B.   On April 21, 1987, Palomar Plaza Assoicates Limited and Tenant
            executed a First Amendment to Lease which providedthat Tenant was
            entitled to the exclusive use of nine (9) parking spaces

       C.   On April 21, 1987, Plaza Associates Limited and Tenant executed a
            second amendment to lease which provided that the initial monthly
            rental rate for Tenants lease of the premises would be $10,500.00

       D.   ON December 23, 1987, Plaza Associates Limited and Tenant executed a
            Third amendment to lease which provided that the term of the lease
            would commence on November 23, 1987 and terminate on November 23,
            2007.

       E.   Effective March 2, 1995, Plaza Associates Limited assigned the lease
            to Palovest Limited partnership, a California limited partnership.

       F.   On or about January 31, 1995, Palovest Limited partnership and
            Tenant executed a Fourth amendment to Lease which provided for a
            reduction in Tenants rent effective May 23, 1995.

       G.   Effective April 17, 1998, Palovest limited partnership assigned the
            lease to Lynn Chuning, an individual. Concurrently therewith, Lynn
            Chuning assigned the lease to Landlord.

       H.   ON December 14, 1998, Community West Bancshares, a California
            Corporation, acquired all of the stock of Tenant without the
            knowledge of consent of Landlord.

       I.   On November 4, 1999, Tenant changed its charter to a state charter
            and changed its name to "Palomar Community Bank."

       J.   Landlord anticipates that 2,985 square feet of space on the second
            floor ("the second floor premises") of the Palomar Bank Building
            will be vacated by the existing tenant on or about July 1, 2001.
            Tennant desires to lease the Second floor premises from Landlord and
            Landlord desires to lease the Second floor premises to Tenant on the
            terms and conditions set forth in the lease except to the extent
            such terms and conditions are expressly modified in this Amendment.

                                       1

<PAGE>

            THEREFORE, in consideration of the convenants, obligations and
            conditions contained herein, landlord and tenant agree as follows:

                                   AGREEMENT

       1.   Second floor premises. In addition to Tenants lease of the Premises,
            Landlord leases to Tenant and Tenant leases from Landlord the Second
            Floor premises. The Second Floor premises is described as follows:
            345 West Grand Avenue, Suite 2, Escondido, California, and is the
            space occupied by the law firm of White and Bright as of the date of
            this Amendment. Tenant's Lease of the Second Floor Premises shall be
            on the same terms and conditions stated in the Lease except to the
            extent expressly modified in this agreement. Unless the context
            otherwise requires, all references in the Lease to the "Premises"
            shall be deemed to include the Second Floor Premises.

       2.   Term. Tenant's lease of the Second Floor Premises shall commence on
            July 1, 2001, or as soon thereafter as Landlord recovers possession
            thereof from the existing tenant and shall terminate concurrently
            with the termination of Tenants's lease of the premises (as defined
            in the lease). Landlord shall use commercially reasonable efforts to
            recover possession of the Second Floor Premises prior to July 1,
            2001.

       3.   Option to Extend. Tenants option to extend the tern of the Lease
            shall include the right to elect to extend the term of Tenants lease
            of the Second Floor Premises as provided in Article 2 of the Lease;
            provided, however, Tenant shall not have the right to elect to
            extend the term of Tenant's lease of the Premises.

       4.   Rent. Tenant shall, without notice or demand, pay to Landlord in
            advance monthly rent for the Second Floor Premises in the sum of
            Five Thousand, Two Hundred Twenty-Three Dollars and Seventy-Five
            cents ($5,223.75) commencing on the Commencement Date. The monthly
            rental amount shall be adjusted annually, commencing May 23, 2002,
            in accordance with the provisions of section 3.1 of the Lease,
            except that the Beginning index shall be the Index which is
            published for the month nearest to July 1, 2001.

       5.   Condition of Premises. Landlord shall deliver the Second Floor
            Premises in broom swept condition. The provisions of Section 6.2 of
            the Lease shall not apply to the Second Floor Premises.
            Notwithstanding the foregoing, Tenant shall have the right at
            Tenants sole cost and expense, to make or cause to be made
            alternations and improvements to the Second Floor Premises provided:
            (a) Tenant shall first obtain Landlord's written approval, which
            approval shall not be unreasonably withheld of the plans and
            specifications for such alternations and improvements. (b) all
            alterations and improvements shall comply with the plans and
            specifications approved by Landlord and with all applicable laws,
            statutes, ordinances or governmental rules or regulations or
            requirements of duly constituted public authorities then in effect,
            (c) all alterations and improvements shall be performed only be
            competent contractors licensed under the laws of the state of
            California and shall be performed pursuant to written contracts and
            (d) such contractor shall have insurance in amounts reasonably
            acceptable to Landlord. Tenant shall defend, indemnify and hold
            Landlord, the property of Landlord and Landlords agents, servants
            and employees, harmless from and against any and all liability,
            claims, costs, and attorney's fees arising out or as a result of
            Tenants's making or causing to be made such alterations and
            improvements.

                                        2

<PAGE>

       6.   Parking spaces. Notwithstanding the provisions of the First
            Amendment to Lease, Tenant shall be entitled to the exclusive use of
            eleven (11) parking spaces.

                  [REMAINDER OF PAGE INTENTIONALLY LEFT BLANK]

                                        3

<PAGE>

                Signature of Landlord to Fifth Amendment to Lease

LANDLORD

CAMP CROOK PROPERTIES, LLC.
A California limited liability company

By:  /s/ Lynn Chuning
     -------------------------
     Lynn Chuning
     It's Managing Member
Date: June 16, 2001

                 Signature of Tenant to Fifth Amendment to Lease
TENANT

PALOMAR COMMUNITY BANK,
A California banking corporation

By:  /s/  Richard M. Sanborn
     ---------------------------
     Richard M. Sanborn
     It's President and CEO

                                        4