Insert a new line in string data

Connectivity LTE Cat-M1 & NB-IoT
1

I am having trouble inserting a new line in my data string. I used the \n as suggested in this forum posting:
Is it possible to include '\n' in SMS-ovef-IP using the Embedded API? - #3 by jwallis
but i am getting an error. if I remove the \n from the data string it works okay. I checked all the documentation I could find and can not locate a solution, please help.

Here is the string I am sending to hologram (using the arduino IDE)
MySerial.write(“{"k":"^n@3jwhX","d":"Hello World \n , Goodbye World ","t":"Fun/and/Games"}\r\n”);

Here is the serial monitor result Note the [2,0] error:
AT+CIPSEND>
{“k”:“^n@3jwhX”,“d”:"Hello World
, Goodbye World ",“t”:“Fun/and/Games”}

SEND OK
[2,0]

Thank you!
oldnotdead

2

Try getting creative with \ characters. Instead of:
\n

also try each of:
\\n
\\\n
\\\\n

and see if any work. Good luck…

3

I tried all three suggestions. The one with \n sends the string with a new line in it,
but is rejected by hologram. I hope there is fix for this. Thank you for your help.
Here are the details

This is the arduino string I am sending with no \n in the data, it is accepted as ok
MySerial.write(“{"k":"^n@vcvwhX","d":"Hello World Goodbye World ","t":"Fun/and/Games"}\r\n”);
this is my serial monitor of what is being sent to Hologram:
AT+CIPSTART=“TCP”,“cloudsocket.hologram.io”,“9999"OK
CONNECT OK
AT+CIPSEND>
{“k”:”^n@3xxxX",“d”:"Hello World Goodbye World ",“t”:“Fun/and/Games”}

SEND OK
[0,0]

this is what all activity show on hologram

Device 1030683
No tags selected. Tags can be added on devices page.
Most recent logs
Search by topics published
KEY
Message sent from Device #1030683
DATA
Hello World Goodbye World
TOPICS
JSONSTRING
Fun/and/Games
2 more…
TRIGGERED ROUTES
first hologram email test

This is the arduino string I am sending with \n in the data, note the [2,0] error ,(rejected by hologram)
MySerial.write(“{"k":"^n@3jxxx","d":"Hello World \n Goodbye World ","t":"Fun/and/Games"}\r\n”);
this is my serial monitor of what is being sent to Hologram:
AT+CIPSTART=“TCP”,“cloudsocket.hologram.io”,"9999"OK

CONNECT OK

AT+CIPSEND>
{“k”:“^n@3jwhX”,“d”:"Hello World
Goodbye World ",“t”:“Fun/and/Games”}

SEND OK
[2,0]

4

Hey sorry I just saw this. Ok I’m bummed that none of the \\\n kind of solutions worked. You have written a program right? …or are you typing in AT commands in real time in the serial monitor?

I’m wondering now if you’re counting the characters correctly. I believe with CIPSEND you have to tell it how many characters you’re sending. So, precisely (I"m using this little message board’s “Preformatted text” feature so the backslashes appear correctly), I’m guessing your code looks like this, with an escaped backslash, followed by an n, so that hologram sees a single backslash, then an n character:

MySerial.write("{\"k\":\"^n@3jwhX\",\"d\":\"Hello World \\n , Goodbye World \",\"t\":\"Fun/and/Games\"}\r\n");

Oh, I don’t think you need the backslash r backslash n at the end. I would remove those. By my count, the above is (including the backslash r backslash n) 76 characters:

{"k":"^n@3jwhX","d":"Hello World \n , Goodbye World ","t":"Fun/and/Games"}rn

I would remove the backslash r backslash n and try sending several times with different character counts to see if one works. I’m not positive this would result in a [2,0] but it definitely might.

Hm, I don’t know where you’re pasting this code from, but it includes some weird “start double quote” and “end double quote” characters. I pasted into my Arduino IDE and it looks like the Arduino IDE does recognize these as different from normal double quote characters, that’s surprising to me. That could theoretically be screwing things up because these chars may take more than 1 byte. Character set issues SUCK. Long shot, but possible. Be sure they’re all “normal double quote” characters in the IDE.

weird start double quote char: “
weird end double quote char: ”
normal double quote char: "

5

I assume you have seen these github repos? If you’re using a SIM7000 cell module I’d check it out.

6

Thanks for the response. I am using the arduino IDE with a sm7000 and an esp32. I tried to keep the program
as simple as possible. I will try your suggestions and hope for the best.

What I found odd, was when using ///n in the data, the return response from hologram .io was correct, it showed the new line, but rejected it with the 2.0 error…

here is what the return looked like.
AT+CIPSTART=“TCP”,“cloudsocket.hologram.io”,“9999"OK
CONNECT OK
AT+CIPSEND>
{“k”:”^n@3jwhX",“d”:"Hello World
Goodbye World ",“t”:“Fun/and/Games”}

SEND OK
[2,0]

After doing more searching, I discovered that arduino ide uses the following characters as excape
characters. I suspect that the backslash escape character is causing the problem, but need to do more
testing.

Here is the list of escape character I found.

SEQUENCE DESCRIPTION REPRESENTATION
' single quote byte 0x27 in ASCII encoding
" double quote byte 0x22 in ASCII encoding
? question mark byte 0x3f in ASCII encoding
\ backslash byte 0x5c in ASCII encoding
\a audible bell byte 0x07 in ASCII encoding
\b backspace byte 0x08 in ASCII encoding
\f form feed – new page byte 0x0c in ASCII encoding
\n line feed – new line byte 0x0a in ASCII encoding
\r carriage return byte 0x0d in ASCII encoding
\t horizontal tab byte 0x09 in ASCII encoding
\v vertical tab byte 0x0b in ASCII encoding
\nnn arbitrary octal value byte nnn
\xnn arbitrary hexadecimal value byte nn
\unnnn (since C++11) universal character name
(arbitrary Unicode value);
may result in several characters code point U+nnnn
\Unnnnnnnn (since C++11) universal character name
(arbitrary Unicode value);
may result in several characters code point U+nnnnnnnn

I will continue with testing and post my results, hopefull i will
have a solution.

oldnotdead

7

Oh man, I just realized the “weird double quote” thing I referenced above wasn’t you at all, it’s this hologram.io message board. I would encourage you not to copy/paste from here into the IDE but to type it into the IDE manually.

So, these escape characters are not part of the Arduino IDE, but part of the C and C++ programming languages. As you have seen, they came from the need for programmers to type “untypeable” characters such as a newline, or audio bell. So they came up with the idea of using a backslash followed by a normal, “typeable” character, for example “backslash a” being the audio bell.

Then of course, they had the problem of how to represent the backslash character, because now it has special meaning. So when the IDE is trying to understand a string such as “this is a string of characters” , when it sees a backslash, it knows that this backslash and the next character will need to be combined to form a single, special character. So - if we just want our string to actually have a backslash character in it - not a special character - you have to have two backslashes in a row. Then the IDE knows OK, the programmer just wants to actually have a backslash character in this string.

Using a double quote character is the same, because as the IDE is reading your string, if it sees a double quote character, it will think it’s the end of the string, but if it sees backslash double quote, it knows OK, the programmer just wants to actually have a double quote character in this string.

How this relates to CIPSEND is that you have to put in the number of characters you’re about to send, right? So if you are sending the string

"this is going to be sent"

then you have to tell CIPSEND that you’re sending exactly 24 characters. But if you are sending the string

"this is going\nto be sent"

while there are literally 25 characters in the string above, the IDE will turn that backslash n into one character so it’s actually only 24. The same is true for all of the escaped characters. A backslash followed by another character is going to only count as 1 character.

I’d bet this character count issue is your issue.

I definitely encourage simplifying what you’d send, i.e. leave off the tag and get this working:

"{"k":"^n@vcvwhX","d":"Hello World"}"

which I think would be

"{\"k\":\"^n@vcvwhX\",\"d\":\"Hello World\"}"

then change to this

"{"k":"^n@vcvwhX","d":"Hello\nWorld"}"

which I think would be

"{\"k\":\"^n@vcvwhX\",\"d\":\"Hello\nWorld\"}"

and don’t change the number of characters that you’re telling CIPSEND you’re sending, which I think is 35. All I’ve done is replace the space character with a newline.

8

Thank you for your detailed response. I tried a few hundered tests this morning using cipsend values between
35 and 40. None of these values worked, the error was [2.0]. I Then tried the same tests using a variety of
\n \\n and \\n, none of these worked. I suspect it is some conflict bbetween escape character \ on the
arduino ide, the hologram.io or possible the sim7000. I am out of ideas. Hope you can help.

Here is the arduino ide skecth I am using for testing.

//HARDWARE IS ESP32 CONNECTED TO SIM7000
#include <HardwareSerial.h>
HardwareSerial MySerial(1);
String inputString=“”;
String incomingString;
void setup() {
MySerial.begin(115200, SERIAL_8N1, 16, 17);
Serial.begin(115200);
MySerial.write(“AT+CIPSHUT\r\n”);
delay(500);
if (MySerial.available() ) //if something is received
{ incomingString = MySerial.readString();Serial.println(incomingString); incomingString = (“”);delay(1000); }

MySerial.write(“AT+CGATT=1\r\n”);
delay(100);
if (MySerial.available() ) //if something is received
{ incomingString = MySerial.readString();Serial.println(incomingString); incomingString = (“”);delay(1000); }

MySerial.write(“AT+CSTT="hologram"\r\n”);
delay(100);
if (MySerial.available() ) //if something is received
{ incomingString = MySerial.readString();Serial.println(incomingString); incomingString = (“”);delay(1000); }

MySerial.write(“AT+CIICR\r\n”);
delay(100);
if (MySerial.available() ) //if something is received
{ incomingString = MySerial.readString();Serial.println(incomingString); incomingString = (“”);delay(1000); }

MySerial.write(“AT+CIFSR\r\n”);
delay(100);
if (MySerial.available() ) //if something is received
{ incomingString = MySerial.readString();Serial.println(incomingString); incomingString = (“”);delay(1000); }

MySerial.write(“AT+CIPSTART="TCP","cloudsocket.hologram.io","9999"\r\n”);
delay(100);
if (MySerial.available() ) //if something is received
{ incomingString = MySerial.readString();Serial.println(incomingString); incomingString = (“”);delay(300); }

MySerial.write(“AT+CIPSEND=36\r\n”); delay(500);
if (MySerial.available() ) //if something is received
{ incomingString = MySerial.readString();Serial.println(incomingString); incomingString = (“”);delay(300); }
incomingString = (“”);delay(30);
MySerial.println(“{"k":"^n@3jwhX","d":"Hello\nWorld"}\r\n”);
MySerial.write((char)26); //must use “cntrl z” (ascii 26) to finalize.
delay(100);
if (MySerial.available() ) //if something is received
{ incomingString = MySerial.readString();Serial.println(incomingString); incomingString = (“”);delay(300); }
}
void loop() {
}

9

So with most IDEs/modems something like \\n will work. The goal is you need to get the individual characters \ and n to show up at our cloud and not the actual line break which might be generated by the compiler seeing \n.
You can also try \\\\n
I think we’ve seen some languages needing that

10

I’ll try playing with it tonight. The key ^n@3jwhX is a security secret, so after we work this problem out you’ll want to regenerate it, but that’s very easy in the dashboard.

11

_

Thank you for your continued interest, and your suggestion about the key. . I have done some more testing, it appears that the \n is appearing
in the string received by hologram.io but is being being ignored. The example below shows that result.
I am using MySerial.write(“AT+CIPSEND\r\n”);, instead of MySerial.write(“AT+CIPSEND=38\r\n”);,

HERE ARE LINES SENT FROM PROGRAM

MySerial.write(“{"k":"^n@3jwhX","d":"Hello World\nGoodbye World"}\r\n”);
MySerial.write((char)26); //must use “cntrl z” (ascii 26) to finalize.

HERE ARE LINES SENT FROM SERIAL MONITOR, note the \n being sent

AT+CIPSTART=“TCP”,“cloudsocket.hologram.io”,“9999"OK
CONNECT OK
AT+CIPSEND> {“k”:”^n@3jwhX",“d”:“Hello World\nGoodbye World”}

SEND OK
[0,0]

THIS IS THE RESULT IN MY HOLOGRAM, dashboard note no \n in data

Device 1030683
No tags selected. Tags can be added on devices page.
Most recent logs
Search by topics published
KEY
Message sent from Device #1030683
DATA
Hello World Goodbye World
TOPICS
JSONSTRING
SOCKETAPI
1 more…

_

12

Sorry I didn’t get to play with it, I was hacking on my own project for several mind-numbing hours last night.

Our communication on this message board is a little challenging because you may notice, at least on my computer, when you type the message in on the left side of the screen, and you type two backslash characters in a row, like I will right now: \ only one shows up on the right side of the screen, and in the message that appears on the message board. It helps to use the “preformatted text” option, which you can do by hitting return a couple times, then starting a line with 4 spaces as I did below. then hit return a couple more times, and that line will automatically become “preformatted” for example

here is preformatted text

and here is normal text below it. Here I’ll type in 2 backslashes: \ and hit return a couple times

then after putting in 4 spaces, I'll do 2 backslashes again: \\ and hit return a couple more times

and with the preformatted text, you can see both of the backslashes show up. Point being, I’m not exactly sure what you’re typing in all the time, and this is certainly one of those times when absolute precision is necessary.

here's some text
and some more
a third line
and a fourth and final line

and back to plain text. Ok, the way to get a LOT of text (like code) preformatted is to type or paste it in here, then highlight all of it, and click the button that looks like </> right above where you type in your message. If you hover your mouse over the button, “Preformatted text” will pop up as a hint.

Anyway, I’ll try to get some time tomorrow to play with your sketch. I have a hard time letting go of problems, ha

1 Like
13

Hi @oldnotdead I took at look at your account and this is working. You’re getting a line break from what I can see in that last message. It’s just the dashboard might not be displaying it but if you were to route the response to somewhere else or if you decode the raw base64 in the API data you’ll see that it’s there.
I think the dashboard may skip line breaks just so it doesn’t break the page, but I’ll mention this to the dashboard team as I can see that this would be confusing

14

Thank you Reuben, glad I didn’t keep digging. I can confirm what Reuben says, I send a lot of messages with newlines and in my dashboard it looks like:
Lockdown: Disabled\nFence: Disabled\nSystem Time: ‘21/04/14,20:02:33-20’

but to the receiver it appears as 3 separate lines.

15

I must be confused about basic hologram functions. I create the simple string i used above. I send The string to the hologram.io. It is received ok in my dashboard, not showing the new line. If i understand the responses i received, it will not be shown in my dashboard, but will be included in the data (base64 encoded], it will also be sent to the receiver.

In my application, i am sending the data to hologram.io email function. The receiver of the mail is not receiving the new line in the message.

Please help.

16

So this is kind of interesting. The newline is sent in the email, but it seems maybe the client might not display it. I did a test to my gmail account and gmail doesn’t display the newline, however, if you click on “Show Original” in gmail, it shows the newline in that case.
Fortunately there does seem to be a fix. You can put the html break character in and in that case gmail and maybe some other clients will show it correctly.
So this message works for me:
{"k":"snip", "d":"asdf<br />\nnewline", "t":"TEST_EMAIL"}

17

Thanks Reuben, i will try it in the AM

18

Thanks Reuben you discovered the solution, I tried it with gmail and comcast.
gmail works comcast does not.
the only change I made to your code was to add the additional escape character backslash:

I appreciate the time and effort you put into finding the solution.