A method for performing a conference call on a server, the method comprising: receiving an inbound communication from each of a plurality of participant devices (120a - 120N) in a conference call communication, each of the plurality of participant devices (120a - 120N) comprising a connection to an accessory device (122a - 122N) having a secure element (124a - 124N) configured to scramble and unscramble audio signals using participant key information stored therein, the inbound communication comprising scrambled media signals (103a - 103N) communicated from the accessory device (122a - 122N) at a participant endpoint of a secure media session (126a - 126N) and relayed by the participant device (120a - 120N); relaying the scrambled media signals (103a - 103N) from each inbound communication to a cryptographic interface with a plurality of server secure elements (110a - 110N) configured to unscramble and scramble audio signals at a server side endpoint of the secure media session (126a - 126N) with a corresponding one of the plurality of accessory devices (122a - 122N) using server key information maintained by the server secure element (110a - 110N) and not accessible by the server; receiving an audio signal (107a - 107N) from the cryptographic interface generated by each of the plurality of server secure elements (110a - 110N) from each scrambled media signal (103a - 103N) received in the media sessions (126a - 126N) with the plurality of participant devices (120a - 120N); mixing the plurality of audio signals to generate conference call data to be communicated to all users as a mixed audio signal; providing the mixed audio signal to the cryptographic interface for each of the plurality of server secure elements (110a - 110N) to scramble the mixed audio signal to generate a plurality of outbound scrambled media signals (105a - 105N); and communicating the outbound scrambled media signals (105a - 105N) as outbound communications to each of the plurality of participant devices (120a - 120N). receiving an inbound communication from each of a plurality of participant devices (120a - 120N) in a conference call communication, each of the plurality of participant devices (120a - 120N) comprising a connection to an accessory device (122a - 122N) having a secure element (124a - 124N) configured to scramble and unscramble audio signals using participant key information stored therein, the inbound communication comprising scrambled media signals (103a - 103N) communicated from the accessory device (122a - 122N) at a participant endpoint of a secure media session (126a - 126N) and relayed by the participant device (120a - 120N); relaying the scrambled media signals (103a - 103N) from each inbound communication to a cryptographic interface with a plurality of server secure elements (110a - 110N) configured to unscramble and scramble audio signals at a server side endpoint of the secure media session (126a - 126N) with a corresponding one of the plurality of accessory devices (122a - 122N) using server key information maintained by the server secure element (110a - 110N) and not accessible by the server; receiving an audio signal (107a - 107N) from the cryptographic interface generated by each of the plurality of server secure elements (110a - 110N) from each scrambled media signal (103a - 103N) received in the media sessions (126a - 126N) with the plurality of participant devices (120a - 120N); mixing the plurality of audio signals to generate conference call data to be communicated to all users as a mixed audio signal; providing the mixed audio signal to the cryptographic interface for each of the plurality of server secure elements (110a - 110N) to scramble the mixed audio signal to generate a plurality of outbound scrambled media signals (105a - 105N); and communicating the outbound scrambled media signals (105a - 105N) as outbound communications to each of the plurality of participant devices (120a - 120N).