Package: discord-ext-voice-recv
discord-ext-voice-recvVoice receive extension package for discord.pyWarningThis extension should be more or less functional, but the code is not yet feature complete.  No guarantees are given for stability or random breaking changes.InstallingPython 3.8 or higher is required, preferably 3.11 or whatever is latestpython -m pip install discord-ext-voice-recvTo install directly from github:python -m pip install git+https://github.com/imayhaveborkedit/discord-ext-voice-recvNaturally, this extension depends ondiscord.pybeing installed with voice support (pynacl).ExampleSee theexample script.Feature overviewCustom VoiceProtocol clientNo monkey patching or bizarre hacks required.  Simply use the library feature to useVoiceRecvClientas the voice client class.  SeeUsage.New eventsThis extension adds the unimplemented voice websocket events and three virtual events.  SeeNew Events.Speaking stateIt is now possible to determine if a member is speaking or not, usingVoiceRecvClient.get_speaking(), or using the speaking events inside anAudioSink.Simple and familiar APIThe overall API is designed to mirror the discord.py voice send API, withAudioSinkbeing the counterpart to the existingAudioSource.  SeeSinks.Convenient included utilitiesBatteries included in the form of useful built inAudioSinks.  Some to match theirAudioSourcecounterpart, some I merely considered useful.  See... uh... TODO.Optional extrasSlightly more complex included batteries that depend on external modules.  These live invoice_recv.extras.  For example,voice_recv.extras.SpeechRecognitionSinkcan be used if the speech_recognition module is available, and can be installed by adding theextrasoptional dependency during install, ex:pip install discord-ext-voice-recv[extras].  More information will be added in the future.More or less typedIt's probably fine.UsageVoiceRecvClientThe classvoice_recv.VoiceRecvClientmust be used inVoiceChannel.connect()to enable voice receive functionality.fromdiscord.extimportvoice_recvvoice_client=awaitvoice_channel.connect(cls=voice_recv.VoiceRecvClient)New voice client functionsdeflisten(sink:voice_recv.AudioSink,*,after=None)->NoneReceives audio data into anAudioSink.  A sink is similar to theAudioSourceclass, where most of the logic is done in a single callback function, but in reverse.  Sinks are explained in detail in theSinkssection below.The finalizer,afteris called after the sink has been exhausted or an error occurred.  The callback signature is the same as the after callback forplay(), one parameter for an optional Exception object.defis_listening()->boolReturnsTrueif the voice client is currently receiving audio.  Specifically, if the bot is reading from the voice socket.defstop()->NoneThis function now stops both receiving and sending of audio.defstop_listening()->NoneStops receiving audio.defstop_playing()->NoneStops playing audio.  This function is identical todiscord.VoiceClient.stop().defget_speaking(member:discord.Member|discord.User)->bool|NoneGets the speaking state (voice activity, the green circle) of a member.  User is typed in for convenience.  Returns None if the member was not found.SinksThe API of this extension is designed to mirror the discord.py voice send API.  Sending audio uses theAudioSourceclass, while receiving audio uses theAudioSinkclass.  A sink is designed to be the inverse of a source.  Essentially, a source is a callback called by discord.py to produce a chunk of audio data.  Conversely, a sink is a callback called by the library to handle a chunk of audio.  Sinks can be composed in the same fashion as sources, creating an audio processing pipeline.  Sources and sinks can even combined into one object to handle both tasks, such as creating a feedback loop.Special care should be taken not to write excessively computationally expensive code, as python is not particularly well suited to real-time audio processing.Due to voice receive being somewhat more complex than voice sending, sinks have additional functionality compared to sources.  However, the core sink functions should look relatively familiar.classMySink(voice_recv.AudioSink):def__init__(self):super().__init__()defwants_opus(self)->bool:returnFalsedefwrite(self,user:User|Member|None,data:VoiceData):...defcleanup(self):...These are the main functions of a sink, names and purpose reflecting that of their source counterparts.  It is important to note thatsuper().__init__()must be called when inheriting fromAudioSink, in contrast toAudioSourcewhich does not have a default__init__function.Thewants_opus()function determines if the sink should receive opus packets or decoded PCM packets.  Care should be taken not to unintentionally mix sinks that want different types.Thewrite()function is the main callback, where the sink logic takes place.  In a sink pipeline, this could alter, inspect, or log a packet, and then write it to a child sink.VoiceDatais a simple container class with attributes for the origin member, opus data, optionally pcm data, and raw audio packet.Thecleanup()function is identical toAudioSource.cleanup(), a finalizer to cleanup any loose ends when the sink has finished its job.Additionally, sinks also have properties for theirclientandvoice_client, as well asparentandchild/childrensinks.Built in SinksThis extension comes with several useful built in sinks, ...
For now justsource dive.  (TODO)Sink event listenersWith AudioSinks being potentially more complex and stateful than AudioSources and the addition of new events, it is sometimes necessary to handle events in the context of a sink.  It would be rather awkward to have to register a sink function withcommands.Bot.add_listener()while dealing with thread safety, and even more so usingdiscord.Client.  To remedy this, listeners can be defined within sinks, similarly to how they work in Cogs.classMySink(AudioSink):@AudioSink.listener()defon_voice_member_disconnect(self,member:discord.Member,ssrc:int|None):print(f"{member}has disconnected")self.do_something_like_handle_disconnect(ssrc)Note that these functions must be sync functions, as they are dispatched from a thread.  Trying to use an async function will result in an error.  This restriction only applies to sink listeners, and normal async event listeners will function as per usual.  The event listener dispatch thread is different from the one used to dispatch thewrite()callback so potential threadsafety issues should be considered.  A decorator argument to run the event callback in the other threadmaybe added later.New eventsasyncdefon_voice_member_speaking_state(member:discord.Member,ssrc:int,state:SpeakingState|int)First and foremost, this event doesNOTrefer to the speaking indicator in discord (the green circle).  For voice activity, seeon_voice_member_speaking_start.
This event is fired when the speaking state (speaking mode) of a member changes.  This happens when:A member first speaks (transmits audio) in a voice, but only once per sessionA member activates or deactivates priority speaker modeThis event is fired once initially to reveal the ssrc of a member, an identifier to map packets to their originating member.  Any packets received from this member before this event fires can (probably) be safely ignored since they are likely just silence packets.asyncdefon_voice_member_disconnect(member:discord.Member,ssrc:int|None)Called when a member disconnects from a voice channel. Thessrcparameter is the unique id a member has to identify which packets belong to them.  This is useful when using custom sinks, particularly those that handle packets from multiple members.asyncdefon_voice_member_video(member:discord.Member,data:voice_recv.VoiceVideoStreams)Called when a member in voice channel toggles their webcam on or off, NOT screenshare.  Screenshare status is only indicated in theself_videoattribute ofdiscord.VoiceState.asyncdefon_voice_member_flags(member:discord.Member,flags:int|None)An undocumented event dispatched when a member joins a voice channel containing a flags bitfield.  Only values0,2, andNonehave been observed so far, but their meaning remains unknown.asyncdefon_voice_member_platform(member:discord.Member,platform:int|str|None)An undocumented event dispatched when a member joins a voice channel containing a platform key, presumably with what platform the member joined on.  Observed values:NoneNo data/unknown0Desktop client/browser1Mobile (android?)The exact meaning of these values is only speculation and assumption.defon_rtcp_packet(packet:RTCPPacket,guild:discord.Guild)A virtual event for when an RTCP packet is received.  This event only works inside of sinks, so it cannot be async.defon_voice_member_speaking_start(member:discord.Member)defon_voice_member_speaking_stop(member:discord.Member)Virtual events for the state of the speaking indicator (the green circle).  These events are synthesized from packet activity and may not exactly match what is displayed in the discord client.  Due to performance issues with asyncio, this event is sink only and cannot be async.Currently missing or WIP features(WIP) Silence generation (pending rewrite)Future plansMuxer AudioSink (mixes multiple audio streams into a single stream)Rust implementations of some components for improved performanceAlternative voice client implementation with a minimal interface intended for use with external data processing