As a follow up, i found out what this was.
I'm doing this on a headless raspberry pi zero. So, i use wireless to SSH into the device.
The default configuration for the dailup scripts in the tutorial have a
defaultroute option, not
replacedefaultroute. This means that the default route is not via ppp0 device after dailup on my system. You can check this by
route -n or
ip route list.
It means that the linux kernel will receive the data, but respond via it's default route, which is not the PPP0 device but (in my case) the wlan0 device. The result is that the SYN/ACK never completes correctly (the ACK is send via teh wrong route).
I fixed this by doing policy based routing:
cat 1000 ppp">> /etc/iproute2/rt_tables
ip route add table 1000 dev ppp0
ip rule add from <ppp0 ip address> table 1000
Hope it helps others! More details can be found at http://lartc.org/howto/lartc.rpdb.html.
An easier fix might be to follow the guide correctly and disable all network devices before starting the ppp device.