PPP, Tunneling, Sending other data, and the generalized system layout of cellular communication

Software Tools Data Engine and Dashboard
1

Let me start by saying I am not a professional coder. This post is a request for guidance.

I’ve gotten mostly through this tutorial:
https://hologram.io/docs/guide/cloud/spacebridge-tunnel/

I downloaded and configured SpaceBridge. Using PuTTy (on a windows 10 machine), I used 127.0.0.1 as my IP address on port 5000 for an SSH connection while SpaceBridge was running. I managed to pull up my Pi’s login credentials for the session before it (slowly) crashed.

I’ve also managed to wrap my head around the CLI commands found generally here:
https://hologram.io/docs/guide/nova/developer-tools/

And sent and received data messages with the dashboard, and sent an SMS to a phone.

Finally, I’ve managed to (sometimes) set up a PPP connection by using the sudo hologram network connect -v command.

I’ve been through a number of the hackster tutorials, but I guess I’m ultimately a little confused on how to set up a data transfer beyond the command line interface.

Am I not deep enough into the AT commands? And what’s with tunneling?

Let me posit the following scenario, as a for instance:

A raspberry pi has a camera that takes a 50kb image once a day, and saves it into a jpeg file in a known folder. What would be the best way (or several ways, or one way?) to inquire over the Nova cellular connection whether the pi has saved an image, then receive that image remotely. Or the same question with the contents of any type of file.

What connection is required (PPP I’m guessing)?
What would be the best way to interface at the Pi (a python/other script running AT commands)?
What would be needed at the computer? Could it be done in-browser? In PuTTy?

Am I correct that PPP is a direct data connection over cellular, and is the preferred method for sending information over the network? (I know this is a super-basic question, but after hours in this forum and going through Hologram’s docs, I’m honestly not sure of the answer.)

Do the CLI commands operate over a PPP connection on the backend? Why does ifconfig show no PPP session when I use the CLI commands but does show one when I sudo hologram network connect? Is it because the CLI commands close the connection right after it is used?

I’d love to see a straightforward tutorial on sending the contents of “some file” from one system (Pi), over Nova cellular, to some other machine at the other end (laptop). Or just a tutorial on sending more complex/ larger sized data in general. I think that would be useful in helping beginners wrap their heads around this technology.

I’m definitely in over my head in understanding network protocols, but I’d like to think it’s possible to figure this stuff out.

Sorry if I’m missing the obvious…

2

Hi, it sounds like you’ve made some progress, but getting to the finish line can be pretty frustrating…

First, you haven’t specified which modem device you are using. Are you using the Nova M (with U-Blox SARA R410M-02B) or something else?

Second, you indicated that you began using WIndows, but are now struggling with a Raspberry Pi. Are you running a Linux variant on the RPi? I’ll assume for the remainder that you are running Linux.
By coincidence, I recently posted the steps I have followed to establish a ppp connection:

https://community.hologram.io/t/connecting-novam-to-cellular-network/2632/22

Several folks have contributed to the thread and now it includes both,

  1. how to set up a ppp connection
  2. discussion about performing data transfers using AT commands (w/o ppp)

including pros and cons of each.

Which way is “best” depends on your goals. I’ll just say that if you set up a ppp connection then at that point your system is connected to the Internet in such a way that individual programs can use the connection transparently. That is, your computer will behave as it does when it has a wifi or Ethernet connection; no individual program will have to do anything beyond create a socket connection or invoke the full array of Linux tools or whatever. The operating system will use the ppp connection to route packets in and out as you would expect.

My understanding is that when you use the Hologram CLI command,

Hologram network connect

under the covers the CLI establishes a ppp connection. And, no, the CLI does not close the ppp connection (unless you run “Hologram network disconnect”). Of course, the connection can close even without any request to close it.

Now, you say you are not a professional coder–nothing wrong with that–but you haven’t indicated your experience level or skill set so it doesn’t make sense to offer specific guidance.

I will make this suggestion: if your goal is to produce a daily 50k image on yourr RPi and move it a remote computer, you must decide:

  1. shall the image file be sent from the RPi to the remote computer (“push” the image from RPi to remote)
  2. shall the remote computer fetch the image from the RPi (“pull” the image by remote from RPi)

If the remote computer is at a known URL (or a fixed ip address) then pushing from the RPi will, generally, be easier. The reason for this is that mobile devices (e.g. RPi with a cellular connection) do not get fixed ip addresses and are generally hidden behind some sort of firewall in such a way that their ip addresses are not easily discovered. if the remote computer is not mobile, it will always be available at a certain URL, so the RPi can easily locate the remote, while the remote cannot locate the mobile RPi.

You will not, without performing magic tricks, be able to connect to your mobile RPi using Putty or other such tools.

If you run ifconfig and you do not see the ppp0 interface then you do not have a ppp connection. Either it was not set up in the first place or it has disappeared. I urge you to take a look at my other posting.

As far as a tutorial, there’s really nothing very special about sending a file. If you want to use AT commands then you must find details in the AT command set documentation (find a link in my other posting). Using the AT commands you can set up a socket connection or perform a file transfer using FTP. If you set up a ppp connection you can transfer files in all the ways available on any Linux system.

3

Thank you for your detailed reply. It really does flesh out the process a bit, particularly with regard to the PPP connection.

Are you using the Nova M (with U-Blox SARA R410M-02B) or something else?

Yes. The NovaM (LTE) model, with NB-IoT turned off.

Second, you indicated that you began using WIndows, but are now struggling with a Raspberry Pi.

The NovaM is attached to a Raspberry Pi 3 B+, running the latest raspbian. In order to operate from the CLI of the headless pi, I SSH into the Pi over wifi (a ios hotspot) from a Windows 10 laptop using PuTTy. (I suspect this might actually be my latest stumbling block-- using wifi to SSH at the same time that I try to get the PPP network activated. I believe I read this in a thread here, but can’t find it now on the forums)

Now, you say you are not a professional coder–nothing wrong with that–but you haven’t indicated your experience level or skill set so it doesn’t make sense to offer specific guidance

I have a few years experience with coding in C/C++, a few in BASIC, I’ve dabbled in PHP and python, and I am just beginning to get into the nitty gritty of the linux/Pi environment. I work as a mechanical engineer and artist, but I would classify my coding skills as amateur.

  1. shall the image file be sent from the RPi to the remote computer (“push” the image from RPi to remote)
  2. shall the remote computer fetch the image from the RPi (“pull” the image by remote from RPi)

Thank you for this. It is something I had not fully thought through. I do have a web server from which I could operate from, so it seems like push may be the best route here, or some sort of handshaking.

I will look back at your other topic and see what I can make work. I was mainly looking for a sort of conceptual explanation of PPP, which you’ve definitely offered here. I did read through it a number of times previously, but I am able to already establish a PPP connection.

A couple of questions/details:

under the covers the CLI establishes a ppp connection. And, no, the CLI does not close the ppp connection (unless you run “Hologram network disconnect”). Of course, the connection can close even without any request to close it.

Is a PPP connection required or used with the sudo hologram send "message"?

I ask this because here is what I see:

  1. Turn on Pi, NovaM starts within seconds, and both LEDS on (blue and red).
  2. Login remotely to CLI with SSH from laptop.
  3. sudo hologram modem operator to check connection. I am already connected to “AT&T Hologram”. (I have not done any sudo hologram connect command at this point).
  4. sudo ifconfig from CLI. I see no PPP connection.
  5. sudo hologram send "help!" sends successfully and I see the message in the dashboard.
  6. Immediately after the send command, I check sudo ifconfig and still see no PPP connection.
  7. Now I sudo hologram network connect and the pi starts up a PPP connection.
  8. sudo ifconfig now finds “PPP0” running.
  9. At this point, sudo hologram send "what about now?" fails with “ERROR: Unable to detect usable serial port”.
  10. sudo ifconfig still shows a PPP0 connection at this point.
  11. sudo hologram network disconnect finds the PPP session and kills the process.
  12. sudo ifconfig confirms no PPP connection present any longer.
  13. And sudo hologram send "And now?" sends successfully to the dashboard.

From that sequence of events, which is repeatable (I did it just now and last night), it indicates to me that the send commands from the CLI either don’t start a PPP connection, or start-send-close a PPP each time.

A guess a larger questions is: Is the send command just a different type of network communication than a PPP connection? Or what’s going on here?

Thanks,
Blake

.

4

I’ll just say that if you set up a ppp connection then at that point your system is connected to the Internet in such a way that individual programs can use the connection transparently. That is, your computer will behave as it does when it has a wifi or Ethernet connection; no individual program will have to do anything beyond create a socket connection or invoke the full array of Linux tools or whatever. The operating system will use the ppp connection to route packets in and out as you would expect.

I also just want to add that this paragraph is immensely helpful to my basic understanding of the processes at work here. I have not found an explanation as clear as this so far anywhere in this forum or in the Hologram help docs. I’m not saying it doesn’t exist, and my apologies if I missed it, but I’ve definitely been looking. Also a possibility: this is knowledge external to the use of the Nova, and I just don’t yet have the requisite understanding of these types of network systems.

I am constantly trying to learn, but I’ve got to start somewhere.

5

Actually, I just found the message about disabling wifi here:
https://community.hologram.io/t/nova-r410-starts-ppp-connection-then-nothing/2498

You may want to disable WIFI and Ethernet so ppp is the only com interface available.

It was actually you, @mbrenner, who said it. I will try this suggestion.

6

Hi Blake,

I started off trying to use the Hologram CLI but eventually threw in the towel. I’m not criticizing that tool, I just didn’t need it and it does add another layer in which almost certainly brings with it bugs. I find it can be quite hard to get to the bottom of problems, and each layer of software increases the debugging challenges. The CLI provides access to functionality I don’t need, so once I learned that “Hologram network connect” really just creates a ppp connection, I realized I can do that myself and by pass CLI.

So, let me state clearly, I am not an expert on the Hologram Python SDK or CLI. My limited understanding is that when the CLI is used to send it is NOT doing setting up a ppp connection to do that.

It seems to me that Hologram has done a very nice job of solving the not-so-easy problem of sending information to a mobile device connected to a Nova. If you need that functionality then using the CLI may be a wise choice. I don’t need that functionality so I haven’t worked in that direction and haven’t learned much about the CLI.

I think you’re right that trying to maintain an ssh connection and set up a ppp connection may be problematic. I’m using an Orange Pi (similar to Raspberry Pi) which has wifi and I find that when I’ve got the wifi on and connected to my router I don’t get a ppp connection to the Nova. The OPi does not have to run headless and has HDMI, so I plug in a monitor and keyboard and, since the OPi is running Ubuntu, it’s pretty much just like using my desktop computer. I compile the program for the OPi on my desktop and then transfer it via FTP over the wireless. After the transfer, I switch from wifi to the Nova if I want to test using the Nova.

I’ll also mention that when I’m on the road and want to do this kind of work I cannot use the wifi for transferring from my desktop (laptop really) to the OPi because I don’t have a configurable router (I’m typically at a hotel). In that situation I use an inexpensive Ethernet switch, connected to both my laptop and my OPi for transferring files. It seems to me you could use such an Ethernet solution for an ssh connection, and then still use your Nova for an Internet connection.

A couple more points:

  1. it can be difficult to tell, in a timely fashion, that a ppp connection has disappeared. After the ppp connection has dropped, it can take minutes before the ppp0 interface disappears from ifconfig.
  2. In point (3) of your “here is what I see” list, you seem are not correctly understanding what “hologram connect” is reporting

Regarding (2)… first, are you able to send AT commands to the Nova and see responses? There are many possible ways to do this. One is to use the Linux “screen” program:

screen /dev/ttyUSB2

(on your system it may be a different dev). Once you can issue AT commands to the Nova, try:

AT+COPS=?

It will take the Nova up to 3 minutes, but eventually it will respond with a list of accessible providers. One of them may show up with a “status” indicating “current.” When you issue, “hologram modem operator” you are getting back the “current” provider. What does it mean? It does not mean that your computer can send data to the Internet. It does mean that your Nova is current registered with that provider. The Nova performs the registration when it powers up. If you are not registered with a provider you will certainly not be able to set up a ppp connection (or use AT commands to move data). But, once registered, you still must set up a ppp connection if you want to use one. That’s why at your step (4) no ppp0 interface appears in ifconfig.

If you are registered, but you are not taking the correct steps to set up a ppp connection you will not be able to move data. At (7) when you do, “hologram network connect,” as you say, it attempts to set up a ppp connection and you’'re then seeing ppp0 in ifconfig at (8).

Obviously, something is going wrong and you learn this in (9). At this point, you can hope and pray for divine intervention, or you can enable debugging and view the log as the ppp daemon is running and while it executes the chat script. I’m saying the logged information will explain exactly what is wrong; the log info can be very cryptic and unclear, but that is where you will find clues.

7

Thank you for your reply.

In point (3) of your “here is what I see” list, you seem are not correctly understanding what “hologram connect” is reporting.

I am aware that being connected to a provider and creating a PPP connection are separate things. But connecting to a provider (getting your red LED), can be problematic in its own right. I wrote that I have not attempted a sudo hologram connect at this point for emphasis, because I’ve seen on other threads people trying to write this command in a flailing attempt to connect to a provider, and because I want to emphasize that no PPP connection was made by me prior to using the hologram send commands.

My steps prove that no, probably no PPP connection is being made with these send commands, or if one is made, it is momentary.

Which seems to be what you think too:

. . .

are you able to send AT commands to the Nova and see responses?

Yes, using Screen, according to these instructions:
https://help.hologram.io/hologram-nova/troubleshooting/sending-at-commands-to-your-nova
It is how I turned off the NB-IoT radio in the first place to help connecting to a network provider (and get the red LED to illuminate).

. . .

I’d still be interested for someone at Hologram to shout out about how the send commands and PPP are related or not related.

Finally, I have managed now to do a file transfer over a PPP connection. Please see this topic, where I have laid out the steps as best I can:
https://community.hologram.io/t/a-simple-out-of-the-box-approach-to-file-transfer-with-novam-solved/2663

Best,
Blake

8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.