Massive Disconnect

#1

I have 20 devices collecting data, which use Nova with Hologram SIMs for connectivity. On 24 Dec 2016 there was some kind of disruption event that hit at least 12 of the 20 units. Of these 12, we contacted people who could unplug/reset them, and 10 of them again established connection (we have not yet diagnosed whether the other 2 have hardware issues). Now, today, I notice on the Hologram dashboard that the remaining 18 have no activity since yesterday (I checked at roughly 15:00): this is a concern since I am used to seeing activity reported once every six hours.

I suspect either it is some kind of Internet link problem (which of course is outside our control) or maybe a more local problem with AT&T 3G. Perhaps also our control in the SBC hosting the Nova is not robust enough to automatically recover, though it has served us well most of the time.

My question to users out there, have you experienced this kind of massive disconnect of multiple devices in production?

#2

Hi @TedHerman I just confirmed with our network ops team that we did not experience any outages at that time. Also our network monitoring is currently not reporting any known issues. If you send the list of affected SIMs to success@hologram.io for our customer success team to follow-up as well.

-Ryan

#3

I’m doing stress test for a prototype the whole month of December until now. I guess I’m in a different region as yours, but my DASH boards also experienced a couple of disconnections… sometimes in GPRS connections but most of the time in the network registration.

So I learned my lesson to include in the code level my own network recovery… parse returned messages and count ERRORS, after 3 counts call recovery block (disconnect - connect to cloud).

Since in PH there’s still 2G services, I found out via NOVA that most of the GPRS disconnection happens when the board shifted from 3G to 2G… so I forced board to Single Mode UMTS (AT+URAT=2).

Also, watch out for the power supply voltage/current… dipping lower than 4.8 volts affected the process of my board… (no response message received via SystemSerial)… and it caused my code to wait for eternity. So better add your millis() expiration in your send/receive code block to make sure your code recovers from this state.

closed #4

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.