Inclusion Criteria:  Have a diagnosis of psoriatic arthritis (PsA) for at least 6 months prior to the first administration of study intervention and meet classification criteria for psoriatic arthritis (CASPAR) criteria at screening Have active PsA as defined by: a) at least 3 swollen joints and at least 3 tender joints at screening and at baseline and b) C-reactive protein (CRP) greater than or equal to (>=) 0.3 milligram/deciliter (mg/dL) at screening from the central laboratory, and despite previous non-biologic disease modifying antirheumatic drug (DMARD), apremilast, and/or nonsteroidal anti-inflammatory drug (NSAID) therapy Have magnetic resonance imaging (MRI)-confirmed PsA axial disease (positive MRI spine and/or sacroiliac [SI] joints, shown by a Spondyloarthritis Research Consortium of Canada [SPARCC] score of >= 3 in either the spine or the sacroiliac joints) Have a Bath Ankylosing Spondylitis Disease Activity Index (BASDAI) score of at least 4, and have a spinal pain score (on a visual analog scale [VAS]) of at least 4 Have active plaque psoriasis, with at least 1 psoriatic plaque of >= 2 centimeter (cm) diameter and/or nail changes consistent with psoriasis, or documented history of plaque psoriasis  Exclusion Criteria:  Has known allergies, hypersensitivity, or intolerance to guselkumab or its excipients Has other inflammatory diseases that might confound the evaluations of benefit of guselkumab therapy, including but not limited to rheumatoid arthritis (RA), ankylosing spondylitis (AS)/non-radiographic-axial spondyloarthritis (this does not include a primary diagnosis of PsA with spondylitis), systemic lupus erythematosus, or Lyme disease Has previously received any biologic treatment Has ever received a Janus kinase (JAK) inhibitor including but not limited to tofacitinib, baricitinib, filgotinib, peficitinib, decernotinib, upadacitinib or any other investigational JAK inhibitor Has received any systemic immunosuppressants (example, azathioprine, cyclosporine, 6 thioguanine, mercaptopurine, mycophenolate mofetil, hydroxyurea, tacrolimus) within 4 weeks of the first administration of study intervention