<PAGE>   1
                                                                     EXHIBIT 4.4

                            LOCK-UP LETTER AGREEMENT

Spanish Broadcasting System, Inc.
2601 South Bayshore Drive
PH II
Coconut Grove, Florida 33133

Dear Sirs:

               Spanish Broadcasting System, Inc. (the "Company"), Rodriguez
Communications Inc. ("RCI") and certain shareholders of RCI (each, a "Seller")
have entered into a Stock Purchase Agreement dated as of May 8, 2000 (the "RCI
Stock Purchase Agreement"), providing for the purchase by the Company of all of
the outstanding shares of common stock of RCI. New World Broadcasters Corp.
("New World") and the Company have entered into an Asset Purchase Agreement
dated as of May 8, 2000 (the "Asset Purchase Agreement"), providing for the
purchase by the Company of Radio Station KTCY-FM (the "Station") and certain
other assets and liabilities related to the Station. The undersigned has or may
receive shares of Common Stock (as defined below) in connection with the RCI
Stock Purchase Agreement and/or the Asset Purchase Agreement.

               In consideration of the execution of the RCI Stock Purchase
Agreement and the Asset Purchase Agreement by the Company and for other good and
valuable consideration, the undersigned hereby irrevocably agrees that, without
the Company's prior written consent, the undersigned will not, directly or
indirectly, for the time periods set forth below, (1) offer for sale, sell,
pledge, or otherwise dispose of (or enter into any transaction or device that is
designed to, or could be expected to, result in the disposition by any person at
any time in the future of) any shares of the Company's Class A Common Stock, par
value $.0001 per share (the "Common Stock"), transferred or to be transferred,
directly or indirectly, to the undersigned, in connection with the RCI Stock
Purchase Agreement or the Asset Purchase Agreement, respectively; provided that
the undersigned may at any time distribute all or a portion of the shares of
Common Stock to an Affiliate (as defined in the RCI Stock Purchase Agreement and
the Asset Purchase Agreement) of the undersigned so long as such Affiliate
agrees to comply with the restrictions contained in this Lock-Up Letter
Agreement (this "Agreement") and the Company has given its prior approval
thereto or (2) enter into any swap or other derivatives transactions that
transfers to another, in whole or in part, any of the economic benefits or risks
of ownership of such shares of Common Stock, whether any such transaction
described in clause (1) or (2) above is to be settled by delivery of Common
Stock or other securities, in cash or otherwise, except in accordance with the
terms of this Agreement (clause (1) and (2) collectively, the "Prohibited
Transfers").

               The undersigned agrees that it may only effect a Prohibited
Transfer (i) with respect to 40% of the aggregate Common Stock held by the
undersigned, after the first anniversary of the Closing Date (as that term is
defined in the RCI Stock Purchase Agreement and the Asset Purchase Agreement)
has passed, and (ii) with respect to the remaining 60% of the





<PAGE>   2


aggregate Common Stock held by the undersigned, after the second anniversary of
the Closing Date has passed.

               In furtherance of the foregoing, you and your Transfer Agent are
hereby authorized to decline to make any transfer of securities if such transfer
would constitute a violation or breach of this Agreement.

               This Lock-Up Letter Agreement supersedes any and all prior
Lock-Up Letter Agreement(s) relating to the RCI Stock Purchase Agreement and the
Asset Purchase Agreement entered into by the undersigned.

               The undersigned hereby represents and warrants that the
undersigned has full power and authority to enter into this Agreement and that,
upon request, the undersigned will execute any additional documents necessary in
connection with the enforcement hereof. Any obligations of the undersigned shall
be binding upon the heirs, personal representative, successors and assigns of
the undersigned.


                                                   Very truly yours,

Date: March 14, 2001                               /s/ James L. Anderson
                                                   --------------------------
                                                   James L. Anderson






                                       2