A solid-state imaging device comprising: a plurality of pixels, each of the plurality of pixels including a photoelectric converter configured to generate charges by photoelectric conversion, a first charge holding unit configured to hold charges generated by the photoelectric converter, a first transfer unit configured to transfer charges from the photoelectric converter to the first charge holding unit, a second charge holding unit (C2; C2, C3) configured to hold charges generated by the photoelectric converter, a second transfer unit configured to transfer charges from the first charge holding unit to the second charge holding unit, and each of the plurality of pixels being configured to output a signal based on charges held in the second charge holding unit; an output line connected to pixels among the plurality of pixels, which are arranged in a column, the signal being output from any one of the pixels arranged in said column to the output line; and a transfer control unit (20, 70) configured to control the first transfer unit to transfer charges generated by the photoelectric converter during one exposure period to the first charge holding unit by a variable number of transfer operations, the variable number being one or greater, characterized by further comprising: an amplifier unit, connected to the output line, configured to amplify the signal output from any one of the pixels arranged in said column by a gain, and a control unit configured to control the gain of the amplifier unit to be a first gain when the number of transfer operations is a first number and to be a second gain different from the first gain when the number of transfer operations is a second number different from the first number, so that a signal level corresponding to a saturation charge amount of the pixel in accordance with the number of transfer operations is included in an input range or an output range of the amplifier unit, wherein the control unit is configured to increase the gain as the number of transfer operations decreases, when the number of transfer operations is smaller than a predetermined number corresponding to a ratio of a saturation charge amount of the first charge holding unit to a saturation charge amount of the photoelectric converter. a plurality of pixels, each of the plurality of pixels including a photoelectric converter configured to generate charges by photoelectric conversion, a first charge holding unit configured to hold charges generated by the photoelectric converter, a first transfer unit configured to transfer charges from the photoelectric converter to the first charge holding unit, a second charge holding unit (C2; C2, C3) configured to hold charges generated by the photoelectric converter, a second transfer unit configured to transfer charges from the first charge holding unit to the second charge holding unit, and each of the plurality of pixels being configured to output a signal based on charges held in the second charge holding unit; an output line connected to pixels among the plurality of pixels, which are arranged in a column, the signal being output from any one of the pixels arranged in said column to the output line; and a transfer control unit (20, 70) configured to control the first transfer unit to transfer charges generated by the photoelectric converter during one exposure period to the first charge holding unit by a variable number of transfer operations, the variable number being one or greater, characterized by further comprising: an amplifier unit, connected to the output line, configured to amplify the signal output from any one of the pixels arranged in said column by a gain, and a control unit configured to control the gain of the amplifier unit to be a first gain when the number of transfer operations is a first number and to be a second gain different from the first gain when the number of transfer operations is a second number different from the first number, so that a signal level corresponding to a saturation charge amount of the pixel in accordance with the number of transfer operations is included in an input range or an output range of the amplifier unit, wherein the control unit is configured to increase the gain as the number of transfer operations decreases, when the number of transfer operations is smaller than a predetermined number corresponding to a ratio of a saturation charge amount of the first charge holding unit to a saturation charge amount of the photoelectric converter. an amplifier unit, connected to the output line, configured to amplify the signal output from any one of the pixels arranged in said column by a gain, and a control unit configured to control the gain of the amplifier unit to be a first gain when the number of transfer operations is a first number and to be a second gain different from the first gain when the number of transfer operations is a second number different from the first number, so that a signal level corresponding to a saturation charge amount of the pixel in accordance with the number of transfer operations is included in an input range or an output range of the amplifier unit, wherein the control unit is configured to increase the gain as the number of transfer operations decreases, when the number of transfer operations is smaller than a predetermined number corresponding to a ratio of a saturation charge amount of the first charge holding unit to a saturation charge amount of the photoelectric converter.