A base station, BS, for handling a carrier aggregation, comprising: at least one storage device; and at least one processing circuit, coupled to the at least one storage device, wherein the at least one storage device stores, and the at least one processing circuit is configured to execute instructions of: connecting to a communication device via a first cell belonging to the BS, and configuring a signaling radio bearer, SRB, to the communication device via the first cell; transmitting a radio resource control, RRC, message to the communication device on the SRB via the first cell, wherein the RRC message comprises a configuration for a second cell to the communication device for the carrier aggregation and comprises a random access channel, RACH, configuration for the communication device to perform a random access, RA, to the second cell, the RACH configuration configures at least one RACH resource, and the RRC message comprises an association configuration associating the at least one RACH resource with a first synchronization signal block, SSB, or with a first channel state information reference signal, CSI-RS, transmitted via the second cell; receiving a RRC response message for responding to the RRC message from the communication device on the SRB via the first cell; receiving a RA preamble from the communication device via the second cell according to the at least one RACH resource; transmitting a RA response, RAR, to the communication device in response to the RA preamble, wherein the RAR comprises a preamble identifier associated with the RA preamble; and communicating with the communication device via the second cell according to the RRC message. at least one storage device; and at least one processing circuit, coupled to the at least one storage device, wherein the at least one storage device stores, and the at least one processing circuit is configured to execute instructions of: connecting to a communication device via a first cell belonging to the BS, and configuring a signaling radio bearer, SRB, to the communication device via the first cell; transmitting a radio resource control, RRC, message to the communication device on the SRB via the first cell, wherein the RRC message comprises a configuration for a second cell to the communication device for the carrier aggregation and comprises a random access channel, RACH, configuration for the communication device to perform a random access, RA, to the second cell, the RACH configuration configures at least one RACH resource, and the RRC message comprises an association configuration associating the at least one RACH resource with a first synchronization signal block, SSB, or with a first channel state information reference signal, CSI-RS, transmitted via the second cell; receiving a RRC response message for responding to the RRC message from the communication device on the SRB via the first cell; receiving a RA preamble from the communication device via the second cell according to the at least one RACH resource; transmitting a RA response, RAR, to the communication device in response to the RA preamble, wherein the RAR comprises a preamble identifier associated with the RA preamble; and communicating with the communication device via the second cell according to the RRC message. connecting to a communication device via a first cell belonging to the BS, and configuring a signaling radio bearer, SRB, to the communication device via the first cell; transmitting a radio resource control, RRC, message to the communication device on the SRB via the first cell, wherein the RRC message comprises a configuration for a second cell to the communication device for the carrier aggregation and comprises a random access channel, RACH, configuration for the communication device to perform a random access, RA, to the second cell, the RACH configuration configures at least one RACH resource, and the RRC message comprises an association configuration associating the at least one RACH resource with a first synchronization signal block, SSB, or with a first channel state information reference signal, CSI-RS, transmitted via the second cell; receiving a RRC response message for responding to the RRC message from the communication device on the SRB via the first cell; receiving a RA preamble from the communication device via the second cell according to the at least one RACH resource; transmitting a RA response, RAR, to the communication device in response to the RA preamble, wherein the RAR comprises a preamble identifier associated with the RA preamble; and communicating with the communication device via the second cell according to the RRC message.