A modular peripheral device assembly (100/400), comprising: a handheld controller configured to sense at least one of a motion and an orientation of the handheld controller, wherein the handheld controller includes, a user input component, a sensor configured to track at least one of the motion and the orientation of the handheld controller, and a first haptic actuator (106a/106b) configured to generate a haptic effect at the handheld controller; an assembly base that includes, an attachment component that is configured to be attachable to and detachable from the handheld controller, and a second haptic actuator (116a/116b) configured to generate the haptic effect at the assembly base or, when the handheld controller is attached to the attachment component, at the handheld controller; and a control unit comprising one or more processing circuits that are located in at least one of the handheld controller and the assembly base, wherein the control unit is configured to determine that the haptic effect needs to be generated for an application receiving input from the peripheral device assembly, characterized in that the control unit is configured to select, based on whether the handheld controller is attached to or detached from the assembly base, at least one of the first and second haptic actuators (106a/106b; 116a/116b) to generate the haptic effect, and configured to activate the at least one haptic actuator (106a/106b; 116a/116b) that is selected to generate the haptic effect. a handheld controller configured to sense at least one of a motion and an orientation of the handheld controller, wherein the handheld controller includes, a user input component, a sensor configured to track at least one of the motion and the orientation of the handheld controller, and a first haptic actuator (106a/106b) configured to generate a haptic effect at the handheld controller; a user input component, a sensor configured to track at least one of the motion and the orientation of the handheld controller, and a first haptic actuator (106a/106b) configured to generate a haptic effect at the handheld controller; an assembly base that includes, an attachment component that is configured to be attachable to and detachable from the handheld controller, and a second haptic actuator (116a/116b) configured to generate the haptic effect at the assembly base or, when the handheld controller is attached to the attachment component, at the handheld controller; and an attachment component that is configured to be attachable to and detachable from the handheld controller, and a second haptic actuator (116a/116b) configured to generate the haptic effect at the assembly base or, when the handheld controller is attached to the attachment component, at the handheld controller; and a control unit comprising one or more processing circuits that are located in at least one of the handheld controller and the assembly base, wherein the control unit is configured to determine that the haptic effect needs to be generated for an application receiving input from the peripheral device assembly, characterized in that the control unit is configured to select, based on whether the handheld controller is attached to or detached from the assembly base, at least one of the first and second haptic actuators (106a/106b; 116a/116b) to generate the haptic effect, and configured to activate the at least one haptic actuator (106a/106b; 116a/116b) that is selected to generate the haptic effect.