After Muir, Robinson went to Pasadena Junior College (PJC). There he continued his sports career by playing basketball, football, baseball, and track. Jackie Robinson was one of the best athletes at Pasadena Junior College. On the football team, he played quarterback and safety. He was a shortstop and leadoff hitter for the baseball team. He broke school broad jump records held by his brother, Mack. As at Muir Hugh School, most of Jackie's teammates were white. While playing football at PJC, Robinson fractured his ankle. Issues from this would later delay his deployment status while in the military. Also while at PJC, he was elected to the Lancers. They were a student-run police group who patrolled various school activities. In 1938, he was elected to the All-Southland Junior College Team for baseball. He was also selected as the region's Most Valuable Player. That year, Robinson was one of ten students named to the school's Order of the Mast and Dagger ("Omicron Mu Delta"). This was awarded to students performing "outstanding service to the school and whose scholastic and citizenship record is worthy of recognition."
An event at PJC shown Robinson's impatience with people he felt were racist. This character trait that would come up several times in his life. On January 25, 1938, he was arrested after he argued against his black friend being taken away by police. Robinson received a two-year suspended sentence. The incident – along with other possible run-ins between Robinson and police – gave Robinson a reputation for getting upset in the face of racial problems. Toward the end of his PJC career, Frank Robinson (to whom Robinson felt closest among his three brothers) was killed in a motorcycle accident. The event motivated Jackie to pursue his athletic career at the University of California, Los Angeles (UCLA), where he could remain closer to Frank's family.
After graduating from PJC in spring 1939, Robinson transferred to UCLA. There he became the school's first athlete to win varsity letters in four sports: baseball, basketball, football, and track. He was one of four African-Americans on the 1939 UCLA Bruins football team. The others were Woody Strode, Kenny Washington, and Ray Bartlett. Washington, Strode, and Robinson made up three of the team's four backfield players. This made UCLA college football's most integrated team. Even though it would be his future career, baseball was Robinson's "worst sport" at UCLA. He hit .097 in his only season, although in his first game he went 4-for-4 and twice stole home.
While a senior at UCLA, Robinson met his future wife, Rachel Isum. She was a UCLA freshman who knew Robinson's sports career at PJC. In the spring semester of 1941, against his mother's and Isum's doubts, Robinson left college before graduation. He took a job as an assistant athletic director with the government's National Youth Administration (NYA) in Atascadero, California.
After the government stopping running the NYA, Robinson traveled to Honolulu in fall 1941 to play football. He played for the semi-professional, racially integrated Honolulu Bears. After a short season, Robinson returned to California in December 1941. There he tried for a career as running back for the Los Angeles Bulldogs of the Pacific Coast Football League. By that time, however, the Japanese attack on Pearl Harbor had taken place. The attack made the United States enter World War II. This ended Robinson's football career.
In 1942, Robinson was drafted and assigned to a segregated Army cavalry unit in Fort Riley, Kansas. Having the necessary requirements, Robinson and several other black soldiers applied for entrance to an Officer Candidate School (OCS) then located at Fort Riley. Although Army policy had allowed black applicants to enter OCS since July 1941, the applications of Robinson and his colleagues were delayed for several months. After protests by boxing champion Joe Louis (then stationed at Fort Riley) and the help of Truman Gibson (then an assistant civilian aide to the Secretary of War), the men were accepted into OCS. This shared military experience created a friendship between Robinson and Louis. Upon finishing OCS, Robinson was commissioned as a second lieutenant in January 1943. Shortly afterward, Robinson and Isum were engaged.
After receiving his commission, Robinson was moved to Fort Hood, Texas. There he joined the 761st "Black Panthers" Tank Battalion. While at Fort Hood, Robinson often used his weekend leave to visit the Rev. Karl Downs, President of Sam Huston College (now Huston-Tillotson University) in nearby Austin, Texas. Downs had been Robinson's pastor at Scott United Methodist Church while Robinson attended PJC.
An event in July 1944 stopped Robinson's military career. While waiting for results of hospital tests on the ankle he had injured in junior college, Robinson got on an Army bus with a fellow officer's wife. Although the Army had its own unsegregated bus line, the bus driver ordered Robinson to move to the back of the bus. Robinson refused. The driver backed down. However, after reaching the end of the line, he got the military police, who took Robinson into custody. Robinson later talked to the duty officer about racist questioning by the officer and his assistant. The officer then recommended Robinson be court-martialed. After Robinson's commander in the 761st, Paul L. Bates, refused to allow this legal action, Robinson was moved to the 758th Battalion. The commander there quickly allowed Robinson to be charged with several offenses, including, among other charges, public drunkenness – even though Robinson did not drink.
By the time of the court-martial in August 1944, the charges against Robinson had been reduced to two counts of insubordination (going against someone in charge of him) during questioning. Robinson was acquitted by an all-white panel of nine officers. Although his former unit, the 761st Tank Battalion, became the first black tank unit to see combat in World War II, Robinson's court-martial stopped him from going with them. He never saw fighting action in the war. After his acquittal, he was moved to Camp Breckinridge, Kentucky. There he served as a coach for army athletics until receiving an honorable discharge in November 1944. While there, Robinson met an ex-player for the Kansas City Monarchs of the Negro American League. The player encouraged Robinson to write the Monarchs and ask for a tryout. Robinson took the ex-player's advice and wrote Monarchs' co-owner Thomas Baird.
After he left the army, Robinson returned to his old football club, the Los Angeles Bulldogs for a short time. Robinson then took an offer from his old friend and pastor Rev. Karl Downs to be the athletic director at Sam Huston College in Austin. The school was part of the Southwestern Athletic Conference at the time. The job included coaching the school's basketball team for the 1944–45 season. As a starting program, few students tried out for the basketball team. Robinson even had to play in some of the practice games. Although his teams were outmatched by opponents, Robinson was respected as a coach who made his players work really hard. He received the respect of, among others, Langston University basketball player Marques Haynes, a future member of the Harlem Globetrotters.