1200 baud data transfer over audio passband of cellphone. Is that possible? - Page 1
Forum section: electronics / projects

author: sv3ora
author_rank: Regular Contributor
timestamp: October 28, 2019, 08:57:34 pm
content: 1200 baud data transfer over audio passband of cellphone. Is that possible through the use of their codecs? Has anyone tried it? There are videos about comms on 300 baud but how about 1200 baud, has anyone tried it?

author: amyk
author_rank: Super Contributor
timestamp: October 29, 2019, 01:34:48 am
content: Tried, yes. Successfully, probably not. GSM audio codecs compress the signal a lot.

author: NiHaoMike
author_rank: Super Contributor
timestamp: October 29, 2019, 01:39:07 am
content: The modern "HD Voice" codecs should give a better chance of working. But if you have that, why not just write an app to interpret the tones on the smartphone itself?

author: coppice
author_rank: Super Contributor
timestamp: October 29, 2019, 02:00:01 am
content: Quote from: sv3ora on October 28, 2019, 08:57:34 pmMost of the speech codecs used for cellular assign a single pitch for each 5ms of speech, which is a fast enough update rate for smooth speech. This is true whether you use the narrow band or the wide band HD voice codecs. However, even the Bell 103 or V.21 modems at 300bps need a pitch update every 3.3ms, so the cellular codecs can't cope. The standard FSK and PSK modems for 1200bps (V.23, Bell 212, V.22) need a pitch update every 1.7ms or 0.83ms, so they are even more heavily corrupted by the speech codecs. You could design a custom modem that will get some data through fairly reliably, by being adapted to the qualities of low bit rate speech codecs. I don't know if anyone has tried to do that.

author: BrianHG
author_rank: Super Contributor
timestamp: October 29, 2019, 03:22:27 am
content: Quote from: coppice on October 29, 2019, 02:00:01 amBeing a digital system, there is a lack of analog noise, so, you might be able to take advantage of that. For example, try a 100hz sine wave or square wave tone with 64QAM levels. At this low frequency, you are relying on the cell networks speech codec just to cope with 100hz, but, modulate the amplitude to 16 levels delivering 4 bits. Scope the in and out, if it works, you have 600 baud. If you get lucky enough, operating on the peaks and troffs of the sine wave with 64QAM, just maybe you can squeeze out 1200 baud. Such a dumb codec shouldn't be too difficult just to try and play with. It should also be easy enough at this point to try 150Hz, 16QAM to achieve the same 600/1200 baud.

author: Psi
author_rank: Super Contributor
timestamp: October 29, 2019, 03:57:11 am
content: 1200 is still pretty slow.I'm sure that is possible if you picked the right codec and optimized for the audio frequency band that are passed by cellphone audio.I wonder how fast you could get with a TCM3105 FSK MODEM IC.People use them on FPV planes to encode a UART onto the audio side channel that 5.8Ghz video transmitters have.

author: sv3ora
author_rank: Regular Contributor
timestamp: October 29, 2019, 09:14:13 am
content: It seems to be a bit of yes and no from people I have asked, both on other forums and here.Is it the "single pitch" problem mentioned in this thread? Is it the heavy compression of some codecs? It is the audio passband?How about trying different mark/space frequencies (which will make the modem non-standard)? How about increrasing the audio volume to distortion levels? How about passing fake voice signals to the passband other than the mark and space frequencies to trick the codec?I think that I will have to test it myself to be sure, as it seems to me not many tries have been made.I will setup two PCs with MixW, set for packet radio. MixW modem, allows the standard rates and mark/space frequencies, but it also allows defining custom rates and mark/space frequencies. So I could play around with these settings and connect the audio on the hands-free jacks of two smartphones.I believe that if it works for me, will work on every country, since the cellphone networks are compatible? (eg. if I visit another country roaming works).Any comments are welcome.

author: ogden
author_rank: Super Contributor
timestamp: October 29, 2019, 09:44:21 am
content: Quote from: coppice on October 29, 2019, 02:00:01 amThen use OFDM modulation with symbol length > 5ms.

author: jhpadjustable
author_rank: Frequent Contributor
timestamp: October 29, 2019, 10:02:30 am
content: Meanwhile, in these first years of the 21st centurymobile phones don't pass "audio". They send a ~13:1 compressed description of audio in 20ms frames, which include the least data necessary to synthesizesomethingthat sounds sort of intelligible and recognizable to a human ear on the far end. They also interpolate any missing packets on the far end, and might dynamically shift to a half-rate version by the network or terminal equipment. These days, it's probably even more complex and more attuned to decoding the human vocal tract. (All the better for mass speech-to-text to hear you, my dear!)Random university PowerPoint PDF illustrating what OP is up against.The academic work in the area is certainly worth a google. Researchers have recently managed to get a"subliminal" channel through the GSM codec. Perhaps not all is lost.I'd suggest a $5 GSM module, if the goal is to get data from point A to point B through the GSM network, rather than an exercise in signal processing and fighting the algorithm.

author: sv3ora
author_rank: Regular Contributor
timestamp: October 29, 2019, 10:19:01 am
content: Sure they are digital! But the end result is the same (or mostly). Every digital system does not carry the real waveform, but a representation of that waveform. But the end result must be the same (or a good representation), if humans need to hear it.I am thinking it that way:Can you pass a single continuous tone of 1200Hz through GSM?Can you pass a single continuous tone of 2200Hz through GSM?Can you pass alternatively switched 1200/2200 tones in low speed through GSM?Can you switch these tones more frequently and how much more?This is the end result and this is what's of interest at the end.

author: TheUnnamedNewbie
author_rank: Super Contributor
timestamp: October 29, 2019, 12:07:21 pm
content: Audio on a cellphone tends to also be compressed in very specific ways that are optimized for voice (which is why the 'please wait' music with callcenters is so horrible). I suspect you will find it very hard to send any data through it as it too will be crazy deformed by the codecs.

author: sv3ora
author_rank: Regular Contributor
timestamp: October 29, 2019, 12:14:19 pm
content: I have encountered this situation you mention, only on remote areas where there is no good signal, or when I used very old phones (1998 era). Dropouts of audio parts or "pixelated" sounds in some audio parts. I have not come across similar situations when there is a good signal.Note, for FSK, audio amplitude compression should be less critical, as it is the frequency of the tones that matter.

author: TheUnnamedNewbie
author_rank: Super Contributor
timestamp: October 29, 2019, 12:27:53 pm
content: Quote from: sv3ora on October 29, 2019, 12:14:19 pmDon't most audio compression algorithms do a lot of stuff in frequency/wavelet domain where frequencies/wavelets that are considered secondary to the main few peaks are just removed?

author: ogden
author_rank: Super Contributor
timestamp: October 29, 2019, 12:30:12 pm
content: Quote from: TheUnnamedNewbie on October 29, 2019, 12:07:21 pmNot true. Speech indeed is spoken *data*. Question here is how to modulate digital data such a way that speech codecs do not affect audio signal during transmission.

author: sv3ora
author_rank: Regular Contributor
timestamp: October 29, 2019, 12:36:00 pm
content: Quote from: ogden on October 29, 2019, 12:30:12 pmPrecisely!I think I am going to do the test with the MixW modem to see what will happen. We may come across pleasant discoveries...or not.

author: coppice
author_rank: Super Contributor
timestamp: October 29, 2019, 01:32:32 pm
content: Quote from: TheUnnamedNewbie on October 29, 2019, 12:07:21 pmYep. Speech codecs are so speech specific that most other things, including multiple voices speaking at the same time, sound awful.

author: coppice
author_rank: Super Contributor
timestamp: October 29, 2019, 01:34:31 pm
content: Quote from: ogden on October 29, 2019, 09:44:21 amWhat does the modulation have to so with the codec being used? They are completely unrelated issues. Most forms of OFDM can add a lot of latency to the channel, but that's about the only specific effect it has on the signal.

author: ogden
author_rank: Super Contributor
timestamp: October 29, 2019, 04:25:55 pm
content: Quote from: coppice on October 29, 2019, 01:34:31 pmYou talked about standard FSK and PSKmodulations, their short symbol length (update every 1.7ms or 0.83ms). I offer OFDM modulation because it has slong(er) symbols that can be stretched depending on FFT order used

author: borjam
author_rank: Unknown
timestamp: October 29, 2019, 05:03:01 pm
content: Old telephone modems based on FSK were the V.21 (300 bps in full duplex) or the asymmetric V.23 (1200/75) intended for videotex systems.The problem with voice compression algorithms is that they degrade the signal in a way that is mostly harmless for speech intelligibility but they can make it impossible for a data modem to work. Especially if you use more complex modulation systems.I remember when some telco in Spain begun using compressors in some of their voice trunks. 28800 bps modems were unable to link beyond 9600 bps.

author: ogden
author_rank: Super Contributor
timestamp: October 29, 2019, 06:01:55 pm
content: Quote from: borjam on October 29, 2019, 05:03:01 pmBS. Many research papers prove you wrong, achieves >1200 bps data transmission rates over GSM-FR (worst case codec, rarely used in modern networks). There are some "secure phone" products that uses not only encrypted VoIP but audio channels as well. You may start here:https://www.researchgate.net/publication/304296823_Data_transmission_via_GSM_voice_channel_for_end_to_end_security

author: coppice
author_rank: Super Contributor
timestamp: October 29, 2019, 06:26:38 pm
content: Quote from: ogden on October 29, 2019, 04:25:55 pmOh, sorry, I was looking at what you said in the wrong way. OFDM won't really help, though. It gets high throughput through parallelism. Good luck getting any parallel signals through a speech codec.

author: ogden
author_rank: Super Contributor
timestamp: October 29, 2019, 07:04:00 pm
content: Quote from: coppice on October 29, 2019, 06:26:38 pmAgreed. I mentioned OFDM just because it kinda solves "too short symbol problem". Modulation shall be tuned forLPC-based speech codec to pass through. Anyway paper I mentioned and many references in it, not mentioning commercial "secure phone" products, proves that it is possible indeed.

author: sv3ora
author_rank: Regular Contributor
timestamp: October 29, 2019, 09:48:55 pm
content: Anyway I have just tested it. I connected a smart phone from it's hands free port to the PC mic and phones. then I called the smart phone from a land line at home.It could not even send 110 baud. It started sending them ok but after 1-3 seconds they were suppressed. Continuously changing the AF volume to 0% and then back to a percentage tricked the codec somehow.Sending pulses of a single tone passed ok.Also, sending a dual tone passed ok.But not FSK at 110, 300 or 1200baud, they all seemed to have this problem I mentioned.Also when sending the above and tried to also send FSK at the same time, again everything was ceased at the channel and nothing heard at the other end, even my sound card was output tones.I have also tried SSTV and the result was that some tines refused to be transmitted.Now, I do not know if this is due to the mobile phone codecs or the landline phone codecs, as I have not tried it with two mobile phones yet.I may have a more detailed analysis on it, maybe note which tones pass ok and which not, or if it is not a tone problem, identify why the codecs cause silence on the channel.

author: borjam
author_rank: Unknown
timestamp: October 29, 2019, 09:54:05 pm
content: Quote from: ogden on October 29, 2019, 06:01:55 pmAmazing work, thanksI agree that I shouldn't say "impossible" but "very hard".

author: ogden
author_rank: Super Contributor
timestamp: October 29, 2019, 10:48:13 pm
content: Quote from: sv3ora on October 29, 2019, 09:48:55 pmLandline usually have 8ksps @8bit PCM, no codecs. Who uses landline today anyway

author: sv3ora
author_rank: Regular Contributor
timestamp: October 29, 2019, 10:50:37 pm
content: So through wired phone there is no compression?

author: ogden
author_rank: Super Contributor
timestamp: October 30, 2019, 12:01:01 am
content: Quote from: sv3ora on October 29, 2019, 10:50:37 pmYou are advised to search The Internet. Answer is all over the place.

author: coppice
author_rank: Super Contributor
timestamp: October 30, 2019, 12:55:01 am
content: Quote from: ogden on October 29, 2019, 10:48:13 pmSimple 8 bit samples would have a rather limited dynamic range for good audio transmission. Landlines have used a simple pseudo-logarithmic compression codec, G.711, since the very beginning of digitising the network in the late 1950s. The variant used in most of the world is called A-law, and the variant used in North America and a few other places is called u-law. They compress linear samples of 13 to 14 bits at 8ksps to samples of 8 bits at 8ksps. This causes significant distortion, but everything designed for the telephone network since the late 1950s has been specifically designed to operate reliably through these codecs, both used alone and in tandem.From the 1980s some operators introduced ADPCM compression for their long distance links, which can handle slow modems, but not the fastest ones. More recently, VoIP has brought some very low bit rate codecs into the mix, which, as with cellular systems, mean all attempts to use standard modems fail. ISDN was supposed to bring the wideband G.722 codec to the telephone network in the late 1980s, but wideband voice didn't catch on until very recently with cellular and VoIP.

author: Psi
author_rank: Super Contributor
timestamp: October 30, 2019, 06:14:30 am
content: i wonder if you could craft the signal so the audio compression algorithm couldn't compress it a huge amount.And get extra bandwidth out of the system that way.Probably not, but its an interesting thought experiment.

author: Siwastaja
author_rank: Super Contributor
timestamp: October 30, 2019, 07:26:36 am
content: Quote from: Psi on October 30, 2019, 06:14:30 amNo, as shown by the earlier replies, GSM compression is a fixed rate (of 13 kbit/s).You'll get the most bandwidth out of the system by carefully designing and optimizing the modulation scheme for the GSM compression.

author: sv3ora
author_rank: Regular Contributor
timestamp: October 30, 2019, 08:41:28 am
content: Quote from: Siwastaja on October 30, 2019, 07:26:36 amThe problem is not the bandwidth, but the compression that seems to be applied by the codecs, which do not let the supposed tones pass for more than a few seconds.

author: sv3ora
author_rank: Regular Contributor
timestamp: October 30, 2019, 08:42:55 am
content: Do you think that I could pass the 1200 baud 1200Hz/2200Hz through the wired telephone instead, or will I have the same problems as the GSM?

author: jhpadjustable
author_rank: Frequent Contributor
timestamp: October 30, 2019, 09:37:33 am
content: Quote from: sv3ora on October 30, 2019, 08:42:55 amWorth a try. The fewer switches and long-haul links your call has to pass through, the better your odds.Quote from: sv3ora on October 30, 2019, 08:41:28 amBandwidth is sometimes informally used to refer to data rate.

author: amyk
author_rank: Super Contributor
timestamp: October 30, 2019, 11:55:44 am
content: It's worth seeking out the official GSM specs (they are dense but free) on codecs and reading them carefully, and also playing around with a PC-based simulator, it will save all the hassle of manipulating actual phones (unless you like that.)

author: _echo
author_rank: Newbie
timestamp: November 04, 2019, 05:26:33 am
content: Hello,Not exactly 1200bps, but if you're trying to get binary data through the audio channel of a cell phone (or any phone for that matter), you might try borrowing the portions of the FSK encoding used by TTYs/TDD. Although their use is declining, this baudot infrastructure is still active today.The core encoding is half duplex FSK, traditionally it's 45.4 bauds/bps. There are a few faster encodings out there, it's possible that those were included.Telecom vocoders usually have hooks to play nice with TTY, some require a menu option to enable, but it's certainly present in most cell phones.Sorry, I don't have any additional details. I thought that it might cause some others to remember the encoding.

author: tooki
author_rank: Super Contributor
timestamp: November 04, 2019, 09:45:26 am
content: Quote from: sv3ora on October 30, 2019, 08:42:55 amShould work fine. AFAIK, the landline network has the ability to detect modems and change something in the codecs to make them work. Fax machines have relied on this for decades.

author: ogden
author_rank: Super Contributor
timestamp: November 04, 2019, 10:04:29 am
content: Quote from: tooki on November 04, 2019, 09:45:26 amReally? Citation needed. My knowledge about landline codec (G.711) say that they do not "detect modems". I talk about landlines of modem/fax era, T1/E1 circuits.

author: GeorgeOfTheJungle
author_rank: Super Contributor
timestamp: November 04, 2019, 10:09:41 am
content: Last time I tried (maybe 10 years ago) I could not fax from my land line anymore. The FAXes are a thing of the 80s!

author: ogden
author_rank: Super Contributor
timestamp: November 04, 2019, 10:48:09 am
content: Quote from: GeorgeOfTheJungle on November 04, 2019, 10:09:41 amYes, so true. Today (cheap) telcos happen to use low rate codecs in their (mostly SIP) long haul trunks. On mobile networks you can literally hear cheap telco operator.BTW I noticed "landline" solution which is made out of cellular/LTE terminal box with landline phone attached. Telco did not even bother to add any kind of UPS function, so your "landline phone" works only when you have mains electricity.

author: tooki
author_rank: Super Contributor
timestamp: November 04, 2019, 11:16:12 am
content: Quote from: ogden on November 04, 2019, 10:04:29 amOh gosh, it’s just a random thing I read or heard somewhere ages ago.I don’t know what the magic search terms are to find out accurately. (Every search result about codecs is talking about VOIP, making it hard to find the docs from before that.) The closest I got is this, which is already VOIP era:https://www.eetimes.com/document.asp?doc_id=1201937Maybe that’s what I heard, but I thought something similar applied to the PSTN, especially for long distance, where heavier compression was used, but fax still worked.Quote from: GeorgeOfTheJungle on November 04, 2019, 10:09:41 amYou would think, right?? But apparently in some countries and/or industries, they’re still widely used, especially in jurisdictions where faxes are considered legal documents, but emails and other internet transmissions are not.I haven’t had a fax-capable device (ignoring my vintage computers with modems) since the 90s when I was living with my parents, so even when I had a landline of my own (which also has been a long time), I never had a chance to try faxing on it.

author: coppice
author_rank: Super Contributor
timestamp: November 04, 2019, 06:20:24 pm
content: Quote from: tooki on November 04, 2019, 09:45:26 amWhy do people make up rubbish like this?

author: tooki
author_rank: Super Contributor
timestamp: November 04, 2019, 10:16:56 pm
content: Quote from: coppice on November 04, 2019, 06:20:24 pmThe possibility of inaccurate recollection (as explained above) is not the same as “making up rubbish”. (And absent documentation either way, I’m not certain that my memory is actually wrong.)Edit: documentation found, thanks to ollopa remembering the specifics. So yeah, I was right. Definitely not "rubbish", and definitely not "made up". (Also, how much more could I have couched it in uncertainty? I made it clear that I wasn't 100% sure, prefacing it with "AFAIK" and only saying that "something" got changed.)

author: ogden
author_rank: Super Contributor
timestamp: November 04, 2019, 10:59:21 pm
content: Quote from: coppice on November 04, 2019, 06:20:24 pmNah. We are all guilty of "imprecisions" here, in this same discussion. No need to start flames.

author: ollopa
author_rank: Regular Contributor
timestamp: November 05, 2019, 12:36:55 am
content: It's not rubbish. One of the early phases of dial-up negotiation is signaling to the channel (AKA phone company) to disable echo cancellation for full-duplex operation. That's completely in-line with "the landline network has the ability to detect modems and change something in the codecs to make them work."

author: westfw
author_rank: Super Contributor
timestamp: November 05, 2019, 11:09:00 am
content: Iirc, the old 1200bps modems required some sort of phase shift modulation, rather than just fsk.(For bi-directional, anyway. The older 1200/150bps modems may have used straight fsk.)

author: tooki
author_rank: Super Contributor
timestamp: November 05, 2019, 11:18:19 am
content: Quote from: ollopa on November 05, 2019, 12:36:55 amThank you! That's what it was.Knowing the specifics, I was able to find more, e.g.ITU-T Recommendation G.16: Interaction aspects of signal processing network equipment, sections 5.2.1–5.2.2, pp 5–9.

author: coppice
author_rank: Super Contributor
timestamp: November 05, 2019, 07:03:34 pm
content: Quote from: tooki on November 04, 2019, 10:16:56 pmWhat ollopa noted is that modems generally signal to disable echo suppressors and echo cancellers. This has nothing to so with the codecs that are used.

author: tooki
author_rank: Super Contributor
timestamp: November 05, 2019, 07:13:26 pm
content: OK, I see what you’re saying. Sorry I chose the word “codec” and didn’t just leave it at “...changes something in how the PSTN handles calls”. I defer to your infinite knowledge and boundless grace and generosity of spirit in this matter...

author: sv3ora
author_rank: Regular Contributor
timestamp: November 14, 2019, 01:10:02 am
content: Well despite said it can't be done, I have managed to pass 1200 baud AFSK (Bell202) through GSM and also between GSM and landline.I have made a small page for it with the experiments I didhttp://qrp.gr/gsmhackI want to investigate how valid is this through out the countries.Would anyone be kind enough to download the file on the page and test it for me in your country's network?To test this yourself, I have created a zip file in the page, which contains two audio wav recordings. One of them is random FSK data at 1200 baud following the Bell202 tones protocol. The other is a single continuous DTMF tone, the number 2. Set up your audio player so it can open multiple instances and play these files at the same time, at about the same audio volume (enable repeat track so that they play continuously). If you do not have an audio jack connection from your phone (hands free plug) to the PC, you could try acoustically coupling the phone to the PC, although I have not tested this. If you can hear the FSK data uninterrupted on the other phone, then it works for you. Note that I have deliberately introduced a few short pauses in the original FSK audio data, as an indication of which part is played, without looking at the screen.

author: jhpadjustable
author_rank: Frequent Contributor
timestamp: November 14, 2019, 08:48:12 pm
content: Holding the channel open with DTMF! Fine business!Have you verified with a proper demodulator yet? What sort of bit error rates are you getting?

author: sv3ora
author_rank: Regular Contributor
timestamp: November 14, 2019, 08:52:37 pm
content: Quote from: jhpadjustable on November 14, 2019, 08:48:12 pmNo, read the last paragraph at the page. The tone seems ok to my ear (not distorted) compared to the original, but hey not tested yet. To do this I will have to built the actual modems. But first I want to see if it is valid in other countries. Can you do this simple test for me?

author: amyk
author_rank: Super Contributor
timestamp: November 15, 2019, 01:00:29 am
content: You can get software that will act as a modem to decode those files. In fact, that's how these work:https://en.wikipedia.org/wiki/Softmodem

author: Tony_G
author_rank: Super Contributor
timestamp: November 15, 2019, 01:14:43 am
content: Saw this thread come up and I wondered if anyone has done TCP over SMS and the first thing to pop up is a software patenthttps://patents.google.com/patent/US8099115B2/enWhat would the world do without them....TonyG

author: sv3ora
author_rank: Regular Contributor
timestamp: November 15, 2019, 09:06:40 am
content: Quote from: Tony_G on November 15, 2019, 01:14:43 amI remember I have seen a girl presenting it but I did not find it very useful as sms costs 30 cents per sms where I live and these are short bursts of data.

author: sv3ora
author_rank: Regular Contributor
timestamp: November 15, 2019, 09:08:02 am
content: Quote from: amyk on November 15, 2019, 01:00:29 amYes I've already used the MixW in packet mode. In fact this is how I generated the audio files. The problem is that I need to build another audio cable to connect the other set of PC/mobile, which I haven't done yet, but eventually I will do.