SendEMAIL using SIM-5320 (3G)  SOLVED - Page 1
Forum section: electronics / projects

author: SL4P
author_rank: Super Contributor
timestamp: August 17, 2016, 06:07:22 am
content: Trying to use current SIMCOM5320modem to end email...Sooo close, but no prize - yet.Unfortunately, it's a bitdifferentto the SIM900, as it implements a newer version of the AT command set.Is there anyone here that can help, or has experience in what I'm missing?Should I revert to the 'old' AT email commands - or will they become extinct within my lifetime!?The reference says the command sequence used below initiates the session, and sends the message - but I'm suspicious that the IP bearer is not being created - but can't find any other thoughts - so 'maybe it does' but my email message ain't going through !Quote

author: hendorog
author_rank: Super Contributor
timestamp: August 17, 2016, 06:32:10 am
content: The manualhttps://cdn-shop.adafruit.com/datasheets/SIMCOM_SIM5320_ATC_EN_V2.02.pdfseems to say you are missing this command:Code:AT+SMTPSENDIt also says the error response from the body is normal when the process of sending the body is ongoing. Just guessing that sending the body just passes the body to the SMTP server and to actually send it you need to use the command above.Worth a try anyway...

author: SL4P
author_rank: Super Contributor
timestamp: August 17, 2016, 06:42:55 am
content: Yeah, sorry..., that's not shown, but is happening down the bottom after the 'day and time' - before the SENT or FAILED outcome.Those response messages are returned from the OK string.Basically, everything throws 'OK', all the way down - including +CSMTPSSEND on the first pass...Which makes me think the 'send' is working, but the modem is sitting on the message - hence the second attempt fails (until a power cycle is completed)Thanks for looking!

author: hendorog
author_rank: Super Contributor
timestamp: August 17, 2016, 06:52:32 am
content: Have you got another SMTP server to test against? Then you can watch the traffic using wireshark or something similar as well as seeing what the server thinks of the conversation.

author: hendorog
author_rank: Super Contributor
timestamp: August 17, 2016, 06:55:49 am
content: Also for the auth case I think you want this:Code:AT+CSMTPSSRV="smtp-relay.gmail.com", 587, 2Source:https://support.google.com/a/answer/2956491?hl=en

author: SL4P
author_rank: Super Contributor
timestamp: August 17, 2016, 07:14:20 am
content: Quote from: hendorog on August 17, 2016, 06:55:49 amTried these as ssugested...QuoteThanks for the thought - same results... first one 'succeeds', second attempt 'fails' - but no email is sent... sigh

author: hendorog
author_rank: Super Contributor
timestamp: August 17, 2016, 07:17:46 am
content: Damn, what about this one:Code:AT+CSMTPSSRV="smtp-relay.gmail.com", 587, 2AT+CSMTPSAUTH=1, "authuser", "authpwd"

author: SL4P
author_rank: Super Contributor
timestamp: August 17, 2016, 07:24:26 am
content: Quote from: hendorog on August 17, 2016, 07:17:46 amSadly not yet...

author: SL4P
author_rank: Super Contributor
timestamp: August 17, 2016, 07:46:52 am
content: Aha!I looked a little deeper into the response messages, and found a result code :::+CSMTPSSEND: 8This indicates a 'Network error'- yet the modem sends and receives voice / sms, and during registration reportsSIM: READYREGISTEREDYES OPTUSWCDMA IMT 2000HSDPAsometimes I getWCDMA 900, but that's neither here nor there.HSDPAis the data channel - registered...So I now have to search & read a bit more... grateful for any more ideas!

author: hendorog
author_rank: Super Contributor
timestamp: August 17, 2016, 07:54:42 am
content: Ah, maybe you need to setup an APN to get internet access? Just like on a mobile phone.

author: hendorog
author_rank: Super Contributor
timestamp: August 17, 2016, 08:03:46 am
content: Optus page says the APN should be "internet"http://www.optus.com.au/shop/support/answer/our-mobile-phone-apns-technical-settings?requestType=NormalRequest&id=1378&typeId=5i.e.Code:AT+CGDCONT=1, "IP", "internet", "0.0.0.0", 0, 0

author: SL4P
author_rank: Super Contributor
timestamp: August 17, 2016, 08:08:34 am
content: I had tested that about 2 minutes before you replied!Here's the latest resultQuote

author: hendorog
author_rank: Super Contributor
timestamp: August 17, 2016, 08:35:16 am
content: HaHere are some more ideas, mostly just guessing tho:Activate and attach the PDP context:Code:AT+CGATT=1AT+CGDCONT=1, "IP", "internet","0.0.0.0",0,0AT+CGACT=1Maybe try without auth again, but on port 587. Port 25 is sometimes blocked on certain networks.TLS can be painful, so worth trying without that.Code:AT+CSMTPSSRV="smtp-relay.google.com", 587, 1AT+CSMTPSAUTH=0Also maybe try this command to clear the settings first:Code:AT+CSMTPSCLEANAlso thought to see if you can leave some of the not needed bearer settings out:Code:AT+CGDCONT=1, "IP", "internet"orAT+CGDCONT=1, "IP", "internet",,,

author: hendorog
author_rank: Super Contributor
timestamp: August 17, 2016, 09:04:01 am
content: Ah, it looks like a typo has crept into the server name during the processit should be:smtp-relay.gmail.comnotsmtp-relay.google.comCouple more tests you could try:DNS test:Code:AT+CDNSGIP="smtp-relay.gmail.com"Ping:Code:AT+CPING="smtp-relay.gmail.com",1

author: SL4P
author_rank: Super Contributor
timestamp: August 17, 2016, 01:38:44 pm
content: Thanks - yes the google/gmail was the victim of some swappin & cutting & pasting!!!But my problem runs deeper...POWER ON -FIRSTattemptQuoteImmediately after - makeSECONDattemptQuoteNote the IP address has gone on the second attempt!I didn't drop the connectionI tried with optus and gmail servers - same resultFinallypower cycle- then aTHIRDattempt - resultidenticalto the FIRSTWhich makes me think the modem isreceivingall the commands - and getting into 'a mode' where it won't accept another 'send' until the previous one is sent, or the modem is power cycled (clearing the operational states)http://www.edaboard.com/thread125473.htmlseems to suggest we're doing the right stuff...The lost IP address may occur because I'm establishing the connection 'again' - but I don't think so.

author: SL4P
author_rank: Super Contributor
timestamp: August 19, 2016, 12:12:01 am
content: More info:After the initial data connection, the modem LED flashed rapidly - indicating 'data mode'Adding +CGATT=0 and +CGACT=0,1 todisablethe data contextafterthe send, released the modem from data mode - and the mode/LED returned to 'non-data mode.Still no success sending email tho', and 255 error persists

author: amyk
author_rank: Super Contributor
timestamp: August 19, 2016, 02:10:55 am
content: Quote from: hendorog on August 17, 2016, 06:52:32 amSeconding this advice. Even just netcat listening on a random port would be sufficient, provided your ISP isn't those "not quite full Internet" ones who don't give you a publicly routable IP. You will at least get an idea of whether it even manages to make a connection to the server.

author: SL4P
author_rank: Super Contributor
timestamp: August 19, 2016, 02:20:07 am
content: The challenge is that the modem is in an embedded controller with no LAN connectivity (it's all 3G).I have tried with other servers and authentication modes with the same result.Basically the connection seems ok, as the dhcp address is issued and attached - as shown.I'm guessing that there is some simple step or oversight that is leaving me cold.I can't find any worked examples of using tnis SSL email configuration on an embedded module.Thanks for the interest.

author: hendorog
author_rank: Super Contributor
timestamp: August 19, 2016, 02:36:31 am
content: Did you try the DNS and Ping tests?That would confirm that firstly the modem can lookup the host name and get the IP, and secondly that it can route traffic to it.Given that there are two 'states' you are seeing I'd try that test in both if you can.

author: SL4P
author_rank: Super Contributor
timestamp: August 28, 2016, 09:44:29 am
content: I've put this one the back burner for now, it was an extra for my own use vs delivering the oroduct...Thanks for the ideas.The ping test didn't do what I expected, so I need to see what else is going wrong in the data setup.cheers

author: janoc
author_rank: Super Contributor
timestamp: August 28, 2016, 10:53:29 am
content: Uh, I second the idea of trying with another SMTP server using the standard port 25. If you want to use GMail, your device must support SSL/TLS, not just simple authentication otherwise Google won't let you connect.I wouldn't be surprised if your modem didn't support the crypto (or you had to do something extra to enable it), as it is not very common on SMTP.

author: SL4P
author_rank: Super Contributor
timestamp: August 28, 2016, 10:58:50 am
content: Thanks guys...I'm pretty sure the problem is at a lower level of connectivity...I've tried with and without SSL/TLS on a variety of servers, always the same error responses.With dotted IP, and FQDN - neither works.As earlier, I get an assigned address, but don't seem to open the IP data path (even though the modem seems to enter data mode)...

author: amyk
author_rank: Super Contributor
timestamp: August 28, 2016, 02:37:18 pm
content: Quote from: SL4P on August 19, 2016, 02:20:07 amIt doesn't need to be on a LAN. You can just set up a very simple server (like the netcat I suggested) on your machine, listening on any port as it seems to be configurable. Then configure on the modem your IP:port as the SMTP server and test to see if it makes a connection and sends something. You can ideally send data from anywhere to anywhere else, that's the idea of the Internet

author: janoc
author_rank: Super Contributor
timestamp: August 28, 2016, 02:50:20 pm
content: Another tool to debug that would be Wireshark (https://www.wireshark.org). However, you would have to make it connect to as server you control and can sniff the connection to.Did you configure APN? Not only told the modem to use the "internet" APN - is it actually set up on that modem? If it isn't (or is misconfigured, unable to reach network, etc.) then it would fail to establish IP connection.

author: SL4P
author_rank: Super Contributor
timestamp: August 28, 2016, 10:40:39 pm
content: Quote from: amyk on August 28, 2016, 02:37:18 pmI don't get this suggestion... No problem to implement a local server, but It's a 3G modem, how's it goingt to reach alocalserver when it can't reach any other server?Same for Wireshark sniffing on a 3G connection (although my experience is very limited - several years ago...)What are you suggesting?Excuse my lack of comprehension here!

author: hendorog
author_rank: Super Contributor
timestamp: August 28, 2016, 10:48:42 pm
content: Quote from: SL4P on August 28, 2016, 09:44:29 amSounds like it's not getting the IP stack setup then - or else the provider is blocking ping packets.I'd be also doing the DNS test as that gives you another data point for comparison. Of course you also want to query the DNS servers from the thing and check that they are working.

author: amyk
author_rank: Super Contributor
timestamp: August 29, 2016, 01:30:31 am
content: Quote from: SL4P on August 28, 2016, 10:40:39 pmThe 3G connection is to the Internet, correct? Your server is also connected to the Internet, and thus they can communicate as long as their know each other's IPs.But as the others here have said, make sure the modem is getting an IP first.

author: SL4P
author_rank: Super Contributor
timestamp: August 29, 2016, 07:15:20 am
content: I thought that's what was being suggested, just (sort of) counterintuitive - adding a number of extra actors in the path...

author: janoc
author_rank: Super Contributor
timestamp: August 29, 2016, 07:10:01 pm
content: Quote from: amyk on August 29, 2016, 01:30:31 amWell, that is why the APN needs to be checked first, whether any traffic actually gets out of it. Wireshask is then for debugging any SMTP-related issues, once you can actually establish the TCP connection.Quote from: amyk on August 29, 2016, 01:30:31 amCareful, 3Gis notthe same as having internet connection! Having 3G connection only means the modem is talking to the wireless base station of the telco. Establishing an Internet connection requires that you have both 3G connectionandthe internet APN correctly configured (and data enabled, but I guess that's default on a modem), otherwise the telco's network will not "put you through" to the Internet and you won't be able to connect to anything.

author: SL4P
author_rank: Super Contributor
timestamp: August 29, 2016, 09:12:05 pm
content: The APN suggestion is becoming more likely, as I'm using a couple of different carrier supplied SIMs, that are 'pre configured" and works fine in *any* data/phone without further configuration (APN from SIM card?).I made the (perhaps false) assumption that the modem would also pick up the APN from the SIM.Will look into that today.... getting an IP address assigned was distracting me from this possibility.

author: janoc
author_rank: Super Contributor
timestamp: August 30, 2016, 09:23:53 am
content: Quote from: SL4P on August 29, 2016, 09:12:05 pmIt may not. Or it could take the wrong one (e.g. the one for wap or MMS services). I had phones that didn't autoconfigure these things and also some carriers don't provide SIMs with the APN preconfigured on them. So better verify it.

author: SL4P
author_rank: Super Contributor
timestamp: August 30, 2016, 09:30:31 am
content: Quote from: janoc on August 30, 2016, 09:23:53 amEven though I didn't change it to get it working - I've added the APN to the configurable params in the calling function... just to be safe in the future. I was trying to reduce the number of customer configured variables - so now there's one more. I love standards that everyone ignores!BTW - I learned that the idea of SSL/TLS email - was to get past the anti-relaying debacle - but it seems the telcos ignore it - so they can force people to subscribe to ther email as well as their carrier service.

author: Wilksey
author_rank: Super Contributor
timestamp: August 30, 2016, 09:37:09 am
content: Ho Hum, the joys of GSM modems.You have to first make sure that the modem is attached to the data channel (AT+CGATT?)Then you have to set the APN (AT+CSTT="apn","username","password")Then you have to open the wireless connections (AT+CIICR)Then you can check your IP to see if you are getting one (AT+CIFSR)I am assuming the command set is the same or very similar to the SIM900.Don't forget to close and shutdown the connection! (AT+CIPCLOSE + AT+CIPSHUT)HTH!

author: SL4P
author_rank: Super Contributor
timestamp: August 30, 2016, 09:42:43 am
content: Quote from: Wilksey on August 30, 2016, 09:37:09 amI wish I'd started out aiming at plain vanilla SMTP, but since the modem supports the 'newer' SSL/TLS secure mail protocols, I wrote around that... slightly different configuration sentences..... and a learning curve! but now it's done!

author: Wilksey
author_rank: Super Contributor
timestamp: August 30, 2016, 09:52:46 am
content: Glad you got it all worked out, care to share the final relevant configuration sections in case anyone else runs into these issues?

author: hendorog
author_rank: Super Contributor
timestamp: August 30, 2016, 10:02:22 am
content: Quote from: SL4P on August 30, 2016, 09:42:43 amAwesome! I'm also keen to see what made it work in the end.

author: SL4P
author_rank: Super Contributor
timestamp: August 30, 2016, 10:06:19 am
content: Here's where it ended up...One critical element is the carriers (more often than not) demand your originating email address is on their domain - I guess that locks in service customers that don't ino enough to work around this stupidity.The original intent was to implement SSL sessions to minimise spam and other eMail nasties, but they turned it into a commercial lock...Here's what works...QuoteIn the setup, the only significant difference I can seeis the order of AT+CGATT and AT+CGDCONTCheers

author: janoc
author_rank: Super Contributor
timestamp: August 30, 2016, 06:45:47 pm
content: Quote from: SL4P on August 30, 2016, 10:06:19 amCool, I am glad you have managed to make it work.The originating e-mail address issue is a trivial prevention of various viruses and malware sending spam, which have pretty much always forged envelope MAIL FROM address. For normal user that shouldn't matter - the envelope address is set to the ISP provided address and any decent e-mail client permits to configure the identity so that a custom From: header appears in the mail for the recipient.Or you mean that the mail service checks the From: headers inside the body of the mail? That would be really strange.The SSL has little to do with spam prevention, actually. The original original idea was to provide a secure encrypted channel for transferring mail (normal SMTP is plaintext, your mail is free for the taking for anyone who happens to have access to the network used to transport it). Just ask Google why they were so pissed and started to convert everything to SSL/TLS when it was revealed that NSA has been sniffing their networks.The anti-spam role mainly came into existence after the proliferation of various viruses and worms spreading by e-mail. The ISPs had to lock their networks down thanks to clueless users, because all it takes is one spammer to get your network blacklisted and suddenly none of your customers can send mail - it will get rejected. Not fun and takes ages to clean up all the while you have irate clients screaming at you. If you ever had to deal with one of the spam blacklists then you will understand.So the ISPs started to block outgoing port 25 (SMTP traffic) from their clients except for their own servers and require authentication before sending (SSL/TLS or POP-before-SMTP). Also many mail servers will refuse mail from servers that are not designated as authorized senders for the given network/domain. Most "mere mortals" don't need to connect anywhere else than to their ISP's e-mail server, so this doesn't limit 99% of the people and cuts down on problems significantly.However, most ISPs will let you send mail (unblock outgoing SMTP traffic) and even run your own mail server if you ask them. They just don't do it by default (way too many clueless users ...). However, do expect having to sign some paperwork about taking responsibility for any spam or other issues in that case. Some will also require you to upgrade to a "business" type of connection.Considering that most people don't even use classic e-mail and e-mail clients today, using SMTP services as a form of ISP lock-in would be completely silly.

author: SL4P
author_rank: Super Contributor
timestamp: August 30, 2016, 11:01:38 pm
content: Quote from: janoc on August 30, 2016, 06:45:47 pmThis is what Ishouldhave written, not specifically spam, but more for general isolation of malware.I chose the SSL compatible client, because of future/forward compatibility, expecting it to help punch past port 25 blocks - as a known, authenticated sender.My expected operation does work in averyfew cases...In an embedded development that can be sold and deployed almost anywhere, it just too hard to 'open' mail routes on every carrier for each of my customers. I'll leave the SSL/TLS capability in there - as part of the manual configuration. The content isn't secret, but I do want it to work in another three years when some other threat locks down everyone's bathroom taps !