<PAGE>


                                                                     EXHIBIT 5.1

                               Baker Botts L.L.P.
                                 One Shell Plaza
                              910 Louisiana Street
                            Houston, Texas 77002-4995

January 31, 2003

Imperial Sugar Company
One Imperial Square
8016 Highway 90-A
Sugar Land, Texas 77478

Ladies and Gentlemen:

     As set forth in the Registration Statement on Form S-8 (the "Registration
Statement") to be filed by Imperial Sugar Company, a Texas corporation (the
"Company"), with the Securities and Exchange Commission (the "Commission") under
the Securities Act of 1933, as amended (the "Act"), relating to up to 1,234,568
shares (the "Shares") of Common Stock of the Company, without par value,
issuable pursuant to the Imperial Sugar Company Long Term Incentive Plan (the
"Plan"), we are passing upon certain legal matters in connection with the
Shares. At your request, we are furnishing this opinion to you for filing as
Exhibit 5.1 to the Registration Statement.

     In our capacity as your counsel in the connection referred to above, we
have examined (i) the originals, or copies certified or otherwise identified, of
the Plan, the Amended and Restated Articles of Incorporation and the Amended and
Restated Bylaws of the Company, each as amended to date, and (ii) the originals,
or copies certified or otherwise identified, of corporate records of the
Company, including minute books of the Company as furnished to us by the
Company, certificates of public officials and of representatives of the Company,
statutes and other instruments and documents as a basis for the opinions
hereinafter expressed. In giving such opinions, we have relied upon certificates
of officers of the Company and of public officials with respect to the accuracy
of the material factual matters contained in such certificates.

     We have assumed that all signatures on all documents examined by us are
genuine, that all documents submitted to us as originals are accurate and
complete, that all documents submitted to us as copies are true and correct
copies of the originals thereof and that all information submitted to us was
accurate and complete.

     On the basis of the foregoing, and subject to the assumptions, limitations
and qualifications set forth herein, we are of the opinion that when issued by
the Company pursuant to the provisions of the Plan following due authorization
of a particular award thereunder by the Board of Directors of the Company or a
duly constituted and acting committee thereof as provided in and in accordance
with the Plan, the Shares issuable pursuant to such award will have been duly
authorized by all necessary corporate action on the part of the Company. Upon
issuance and delivery of such Shares from time to time pursuant to the terms of
such award for the consideration established pursuant to the terms of the Plan
and otherwise in accordance with the terms and conditions of such award,
including, if applicable, the lapse of any restrictions relating thereto, the
satisfaction of any performance conditions associated therewith and any
requisite determinations by or pursuant to the authority of the Board of
Directors or a duly constituted and acting committee thereof as provided
therein, and, in the case of stock options, the exercise thereof and payment for
such Shares as provided therein, such Shares will be validly issued, fully paid
and nonassessable.

     We hereby consent to the filing of this opinion with the Commission as
Exhibit 5.1 to the Registration Statement. In giving such consent, we do not
admit that we are within the category of persons whose consent is required under
Section 7 of the Act or the rules and regulations of the Commission thereunder.

                                          Very truly yours,

                                          BAKER BOTTS L.L.P.