Problem with new Virgin 50Mbps Modem

[Alain Williams]

You may be interested in the solution for a customer of mine who recently
installed (upgraded to) the Virgin 50Mbps Super Hub.

The symptoms were lost packets, this badly affected incoming emails (we run an
MTA) especially, for some reason, those from google/gmail. There was (I think)
also a problem with an SSL session.

The fix is to set the MTU to 1460. We did that with an immediate improvement.

The sad thing is that we have been beating our heads on the table for a week,
with various Virgin support droids blaming our config.

On a redhat system edit /etc/sysconfig/network-scripts/ifcfg-eth0 and
insert the line:
MTU=1460

After fixing it you will see 1460 as in below:

# ifconfig eth0
eth0 Link encap:Ethernet HWaddr 00:0D:56:64:3A:C6
inet addr:192.168.10.2 Bcast:192.168.10.255 Mask:255.255.255.0
inet6 addr: fe80::20d:56ff:fe64:3ac6/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1460 Metric:1
RX packets:31171623 errors:4 dropped:13 overruns:0 frame:0
TX packets:19810324 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:4150314049 (3.8 GiB) TX bytes:3814195185 (3.5 GiB)
Interrupt:201

[James Courtier-Dutton]

Post by Alain Williams
You may be interested in the solution for a customer of mine who recently
installed (upgraded to) the Virgin 50Mbps Super Hub.
The symptoms were lost packets, this badly affected incoming emails (we run an
MTA) especially, for some reason, those from google/gmail. There was (I think)
also a problem with an SSL session.
The fix is to set the MTU to 1460. We did that with an immediate improvement.
The sad thing is that we have been beating our heads on the table for a week,
with various Virgin support droids blaming our config.
On a redhat system edit /etc/sysconfig/network-scripts/ifcfg-eth0 and
   MTU=1460
# ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 00:0D:56:64:3A:C6
         inet addr:192.168.10.2  Bcast:192.168.10.255  Mask:255.255.255.0
         inet6 addr: fe80::20d:56ff:fe64:3ac6/64 Scope:Link
         UP BROADCAST RUNNING MULTICAST  MTU:1460  Metric:1
         RX packets:31171623 errors:4 dropped:13 overruns:0 frame:0
         TX packets:19810324 errors:0 dropped:0 overruns:0 carrier:0
         collisions:0 txqueuelen:1000
         RX bytes:4150314049 (3.8 GiB)  TX bytes:3814195185 (3.5 GiB)
         Interrupt:201

Hi,

Just so you know, the cause is a problem with their network, not yours.
The problem you were running into is call "black hole" packets.
This is where, if you send packets of varying sizes, some will not get through.
What you did is work around the problem, not actually fix it.

What should happen is this:
For all packet sizes up to a fixed limit, packets with the DF bit set
should get through.
For any packets sizes larger than that fixed limit, you should get a
ICMP packet back saying the packet was too big.

The problem with their network is that there are black holes.
I.e. Packets of certain sized get lost, instead of "get through" or
"get ICMP response".

The problem packet sizes in your case, are somewhere between 1460 and 1500.
The problem could be anything on their network, the new home hub, or
some device in their network.

Kind Regards

James
--
Gllug mailing list - ***@gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug

[Alain Williams]

This post might be inappropriate. Click to display it.

[James Courtier-Dutton]

Post by Alain Williams

Post by James Courtier-Dutton
Hi,
Just so you know, the cause is a problem with their network, not yours.

Yes ... they did (eventually) admit that.
''My'' network is at a school, I have fixed the Linux servers, but there are
many other machines - including the admin block which is staunchly MS owned.
At the center of the network is a Linux box (called Hermes) with 4 ethernet
interfaces, one of which is the only connection to the Virgin modem.  I am
looking to see if I can fix it by tickling the Hermes firewall. I note the 2
items below and wonder if anyone has any experience with this ?
  TCPMSS
      This target allows to alter the MSS value of TCP SYN packets, to
      control the maximum size for that connection  (usually  limiting
      it  to  your  outgoing interfaces MTU minus 40).  Of course, it
      can only be used in conjunction with -p tcp.  It is  only  valid
      in the mangle table.
      This  target  is  used  to overcome criminally braindead ISPs or
      servers which block  ICMP  Fragmentation  Needed  packets.   The
      symptoms  of  this  problem  are that everything works fine from
      your Linux firewall/router, but machines  behind  it  can  never
       1) Web browsers connect, then hang with no data received.
       2) Small mail works fine, but large emails hang.
       3) ssh works fine, but scp hangs after initial handshaking.
      Workaround: activate this option and add a rule to your firewall
       iptables -t mangle -A FORWARD -p tcp --tcp-flags SYN,RST SYN \
                   -j TCPMSS --clamp-mss-to-pmtu
      --set-mss value
             Explicitly set MSS option to specified value.
      --clamp-mss-to-pmtu
             Automatically clamp MSS value to (path_MTU - 40).
      These options are mutually exclusive.
   https://blue-labs.org/howto/mtu-mss.php

Post by James Courtier-Dutton
The problem you were running into is call "black hole" packets.
This is where, if you send packets of varying sizes, some will not get through.
What you did is work around the problem, not actually fix it.

My advice, don't touch the tcp mss values at the firewall. It fixes
the symptom for TCP packets, but does nothing for UDP packets. I.e.
Some media streaming and P2P protocols will still be broken.

Until Virgin fix their stuff, just lower the MTU on the firewall
interface next to the Virgin Hub.
This will cure the symptom for all hosts at your site.
Once Virgin fix their stuff, restore the MTU value to its default.

Good to hear they you have your own firewall.

If you are interested in why you see the symptoms you see. The main
reason is that "black hole packets" break path MTU discovery (RFC1191)

Kind Regards

James
--
Gllug mailing list - ***@gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug

[Christopher Hunter]

This post might be inappropriate. Click to display it.

[Alain Williams]

Post by Christopher Hunter
As ever, my advice remains the same - if possible, find ANY other way to
get connected - Virgin are a clueless bunch of clowns who overcharge for
a fundamentally (and deliberately) broken service.

I know ... unfortunately geography dictates use of Virgin (too far from the
exchange to get a decent speed over the 'phone line).

[Jason Clifford]

Post by Alain Williams
I know ... unfortunately geography dictates use of Virgin (too far from the
exchange to get a decent speed over the 'phone line).

This is the case in many places where Virgin operate. Do you know
whether there are plans for FTTC in that area?

As the only copper if from the street cabinet to the customer premises
you get much better rates with FTTC.


--
Gllug mailing list - ***@gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug

[James Courtier-Dutton]

Post by Christopher Hunter

Post by Alain Williams
You may be interested in the solution for a customer of mine who recently
installed (upgraded to) the Virgin 50Mbps Super Hub.
The symptoms were lost packets, this badly affected incoming emails (we run an
MTA) especially, for some reason, those from google/gmail. There was (I think)
also a problem with an SSL session.
The fix is to set the MTU to 1460. We did that with an immediate improvement.
The sad thing is that we have been beating our heads on the table for a week,
with various Virgin support droids blaming our config.
On a redhat system edit /etc/sysconfig/network-scripts/ifcfg-eth0 and
    MTU=1460
# ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 00:0D:56:64:3A:C6
          inet addr:192.168.10.2  Bcast:192.168.10.255  Mask:255.255.255.0
          inet6 addr: fe80::20d:56ff:fe64:3ac6/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1460  Metric:1
          RX packets:31171623 errors:4 dropped:13 overruns:0 frame:0
          TX packets:19810324 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:4150314049 (3.8 GiB)  TX bytes:3814195185 (3.5 GiB)
          Interrupt:201

I'm very surprised that this has only just made it to this list - it's a
problem that's been around on Virgin ("on the ridiculous") since they
"upgraded" the network they inherited from Telewest.
One thing that hasn't changed - they don't understand the problem and
won't accept that there could be anything wrong with their network.

This is not just a Virgin problem. I have seen the problem present on
Cable&Wireless and Global Crossing (Level 3) WAN networks. The
difference there is at least they fix the problem when you tell them
what it is.
The problem can be very difficult to spot, as you can get black hole
packets on only certain paths, and also only in one direction. So, for
a WAN link, you own both directions so can send packets from each end,
but for Internet, you only own one end. The only way to send packets
from the other end is if you have a person across the internet who can
help, or a host on the internet that can help.
For example, if you told me your IP address, I could tell if there was
a black hole from me to you, but not the other way round. That is also
assuming that I can do ping (icmp echo) requests to you, although it
can be tested with UDP and TCP packets if needed.
--
Gllug mailing list - ***@gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug