Davidiw:NSDI10

From Grid-Appliance Wiki

Notes:

• Abstract - 5 PM PDT, September 25
• Paper - 5 PM PDT, October 2
• 14 Pages, double column

Contents 1 Overview 2 Tasks 3 Outline 4 Tasks Indepth 4.1 Chord P2PVPN 4.2 Pastry Comparison 4.3 Subring 4.4 Improving Tunnels and Proxying 4.4.1 Tunnels 4.4.2 Proxy Tunnels 4.4.3 Switching Between Tunnels and Direct Connections 4.5 OpenVPN 4.6 P2PVPNs 4.6.1 Wippien 4.6.2 Review of Some Current P2P VPNs 4.7 P2P VPN Gateway 4.7.1 High Level 4.7.2 Routing rules 4.7.3 Raw Packets 4.8 Managing System 4.9 Setting up a Linux Gateway 4.9.1 The Server 4.9.2 The Client

Overview

Motivation - In recent years, P2P VPNs have become quite popular by allowing users to connect directly with each other bypassing the overhead of communication through a third party proxy. These P2P VPNs still require connecting to a central server for authentication, NAT traversal, and proxying in the off chance NAT traversal fails. While this significantly improves upon classical, centralized VPNs, this approach now requires maintenance of all-to-all connections during run-time or involve the centralized authentication entity for each live connection attempt. For this solution, we propose a completely run-time decentralized P2P model. In this paper, we will describe the components of this model, describe our reference implementation, and evaluate the reference implementation. A decentralized P2P VPN reduces the requirements for connectivity, offers better proxy selection when NAT traversal is not available, and provides an opportunity for more intuitive trust solutions. For evaluation, we will compare system and networking overheads of the different VPN technology focusing on latency, bandwidth, CPU, and memory.

Tasks

• Implement an OpenVPN (Classical, Centralized VPN) server in a VM -- Linton
• Implement a Wippien (Centralized P2P VPN) server in a VM -- Yonggang
• Port IPOP to another P2P infrastructure, perhaps NChord (C# Chord), motivation for Brunet -- Jiangyan
  • Easy to make a VPN
  • Poor configurability (i.e. no point in routing via overlay, thus no NAT traversal)
• Subring based upon groups for broadcast, multicast, routing -- Pierre
• System management, crawling both linear and MapReduce - Kyungyong
• Implement a P2P model that provides P2P bypass (when available) in a generic fashion -- for performance purposes -- David
• Investigate potentially distributivity of CiscoVPN (and the like)
• Citation for centralized VPN issues
• Providing an external gateway
• Setup an environment to test cost per connection of different VPN solutions
• Get proxy-Chota working
• Intelligent algorithm for maintaining Chota
• Look at ring and find packet distribution, make intermediate shortcuts, adaptive paths
• Examine wireless literature for overlap
• Artificial network constraints in EC2 -- Emulab, NistNAT, Havok Grid 5000

Outline

1. Introduction
2. VPN Background
   1. Approach
      1. Centralized VPNs
      2. Centralized P2P VPNs
      3. P2P VPNs
   2. Models
      1. Private overlay
      2. Gateway / Security
3. Overview of structured P2P systems
   1. Chord
   2. Brunet
4. Binding a VPN to a P2P structure
   1. Security quirks
   2. Private overlay model
   3. Gateway model
   4. Group ring for efficient multicast and low latency communication
   5. Efficient Chota handling
   6. Proxy, intermediate shortcuts, adaptive paths for failed NAT cases
   7. P2P model for bypassing P2P
5. Evaluation
   1. Bandwidth, latency, CPU
   2. Connection
   3. Proxying comparison
6. Conclusions

Tasks Indepth

Chord P2PVPN

• Use Chord to look up IP address:port and plug into "IPOP Lite".
  • 2 Lookups
  • First look up the P2P ID from DHT by the virtual IP read from Tap
  • Then look up the real/physical IP from overlay by the P2P ID
  • Direct communication to PhysicalIP:port, the port is unchanged.
• Portability of the idea
  • After moving to new IP, the peer can still be found by its virtual IP.
• Necessity for Brunet and recursive overlay communication
  • Chord has to operate on public IPs, no NAT traversal mechanisms.
  • Only has direct communication. Lookup latency.
• Status quo of NChord
  • Only a sample implementation of DHT, minimal replication and fault tolerance attempt.
  • Implemented using .NET Remoting. No special ports for DHT communication.
• Recursive routing on NChord?
  • FindSuccessor is recurse.
  • Current implementation of DHT operations: recursively find the successor and then call it directly.
  • Need to implement recursive RPC by ourselves.

Pastry Comparison

• FreePastry, in Java, can be used via IKVM.
• Network Locality: the number of IP routing hops or geographic distance vs. Brunet's Network Coordinates.

Subring

RingLeafEdges:

• In the relative main rings dht (subring:namespace), store RingLeafEL TA for remote nodes, they consist of the main rings namespace and remote nodes address
• In the subring, when CreateEdgeTo is called, it will return back a Edge that uses an AHSender for the main ring
• Question is how do we keep this to only LeafConnectionOverlord, perhaps we can some push mechanism into the LCO for it to work.

Improving Tunnels and Proxying

Tunnels

Can easily support one tunnel in the path: Consider A, B, C, and D

• The following connections are possible:
  • A can connect to C, D
  • B can connect to D
  • C can connect to A
• In the current state C and B could not connect, they do not have a direct connection
• By allowing only a single tunnel in the direct path though, C and B can connect via:
  • C forwards packet to A who tunnels through D to B
  • C tunnels to D through A and D forwards packet to B
  • B forwards packet to D who tunnels through D to A
  • B tunnels to A through D and A forwards packet to C
• Does this introduce any new potential errors? I don't see any....

Proxy Tunnels

• Improve NCService so it makes some sense and is testable
• Proxy tunnel algorithm for two unconnected peers, A and B
  • Initiated by A after chota fails
  • A sends to B, via overlay, a request for low latency neighbors
  • B sends to A the results
  • A connects to one or more of these low latency neighbors
  • A forms a proxy-tunnel edge via this to B
  • B returns the favor and then we're golden

Switching Between Tunnels and Direct Connections

OpenVPN

Multiple servers

• Client chooses servers either by round-robin or linearly. This requirement has to be explicitly stated in the client config file along with the servers information.
• How do Servers communicate with each other? How do Clients communicate with each other?
  • Documentation on this seemed to err a little. So I set up my own tes=== OpenVPN ===

Multiple servers

• Client chooses servers either by round-robin or linearly. This requirement has to be explicitly stated in the client config file along with the servers information.
• How do Servers communicate with each other? How do Clients communicate with each other?
  • Documentation on this seemed to err a little. So I set up my own test bed with four OpenVPN servers and had eight OpenVPN clients connect to them in round-robin. Tests clearly show that OpenVPN does not dynamically load balance or ever configure multiple servers to communicate with each other. Client daemons could be setup to connect to server daemons through a secure virtual tunnel. Clients connected to different servers cannot communicate with each other at this stage. In order for one OpenVPN server to see another, a TUN connection has to be explicitly setup. Through this and only this can now the clients communicate.
• Steps to add a new server to the system?
  • Adding a new server to a system seems as easy as adding the IP and PORT information of the new server to all the Clients configuration files. For load balancing and other communicative needs, a TUN device has to be set up among the servers. Then the routing tables have to be modified so that all the clients and the servers are now aware of each other.

Client-to-client communication

• Is it encrypted so that the server can't see the packet?
  • OpenVPN supports two authentication modes

1. Static Key -- Use a pre-shared static key
2. TLS -- Use SSL/TLS + certificates for authentication and key exchange

• In static key mode, a pre-shared key is generated and shared between both OpenVPN peers before the tunnel is started. This static key contains 4 independent keys: HMAC send, HMAC receive, encrypt, and decrypt. If this mode is used, then the server cannot decrypt the clients packets.

• In SSL/TLS mode, an SSL session is established with bidirectional authentication (i.e. each side of the connection must present its own certificate). If the SSL/TLS authentication succeeds, encryption/decryption and HMAC key source material is then randomly generated by OpenSSL's RAND_bytes function and exchanged over the SSL/TLS connection. Both sides of the connection contribute random source material. In this mode, the server can decrypt the clients paclets

• Enabled by default?

Either one of the above authentication mode can be easily chosen by changing the server's congfiguration files. But no SSL is the default mode, which does not hide packet info from the servers.

• Locally settable option?

Yes. It can be set in the config file of the two clients wishing to communicate privately. But 'client-to-client' flag has to be set in the server's config file and all clients have to belong to the same server or the routing tables appropriately modified. t bed with four OpenVPN servers and had eight OpenVPN clients connect to them in round-robin. Tests clearly show that OpenVPN does not dynamically load balance or ever configure multiple servers to communicate with each other. Client daemons could be setup to connect to server daemons through a secure virtual tunnel. Clients connected to different servers cannot communicate with each other at this stage. In order for one OpenVPN server to see another, a TUN connection has to be explicitly setup. Through this and only this can now the clients communicate.

P2PVPNs

Wippien

1. Wippien is mainly built for Windows, the Linux version is very generic. Two Linux machines can't establish VPN (By grabbing the packets, I saw the NAT traversal doesn't start up. That's why even if I use two machines with public IPs, they can't establish VPN). If two machines want to establish VPN, at least one should be Windows.
2. Wippien is open source, but the source code depends on the wodVPN library (released by WeOnlyDo Inc.) which is not open source (so does the Linux version). Because we don't have the library, we can't do the compilation. So we can't recompile the Linux version to improve it, like changing the mediator address.
3. Wippien needs one jabber account for each VPN node, because Wippien will use the exact jabber account to generate virtual IPs for the users, so using the same account at multiple locations may result in having the same Virtual IP. Mediator does the Virtual IP generation; we can have our own mediator, but we can't direct Linux Wippien client to our mediator because this configuration is written into Linux client source code.
4. I asked the Wippien developer if Wippien supports proxy tunneling. They said no, Wippien only supports direct connections. So Wippien only supports NAT traversal; it doesn't support proxy tunneling.

Review of Some Current P2P VPNs

Name	Platform	Free	client limit	server creation	Internet Tunneling	Data Relay	bootstrap	Note
Remobo	win / linux(gtk+)	yes	unlimited, dup login	can't	no	yes	connect to server	can't run on command line
neorouter	c: win, s: win/linux	yes	unlimited	can	no	via router	connect to vendor server	router is the proxy for routing to other clients
hamachi	win / linux	yes	16	can't	no	yes	connect to server
N2N	win / linux	yes	unlimited	can	no	yes, via super node	connect to super node
UIA	win / linux	yes	unlimited	no server	no	yes	?	It's not VPN, P2P network
P2PVPN	win / linux(32bit)	yes	unlimited	no server	no	no	connect to server	no NAT / firewall traversal
leafnetworks	win	yes	unlimited	can't	no	no	connect to server
LAN Bridger	win	yes	unlimited	can	?	?	connect to the server (calledhub)
Gbridge	win	yes	unlimited	can't	no	yes	connect to google and Gbridge server	extension of Google network service
vnn	win	yes		no server				only supports TCP/IP

P2P VPN Gateway

High Level

• Usual case:
  • Path - IP <-> Gateway <-> Internet
  • Packet - Data | IP to Internet | Ethernet to GW
• VPN Full Tunnel
  • Path IP <-> Gateway <-> Internet <-> VPN Gateway <-> Internet
  • Packet - Encrypted(Data | IP to Internet) | IP to VPN GW | Ethernet to GW

Routing rules

One method involves adding a routing rule for each endpoint.

• Swap the local gateway with the VPN gateways address
• Use a TAAuthorizer to intercept CreateEdgeTo calls
  • One TAAuthorizer will add a routing rule
  • Another must verify we aren't adding a VPN IP to the Routing table, or it will screw everything up!
  • We must clean the table up or it can become loaded, simply reset the TAAuth for the edge listener and it will iterate through all active edges!
• The Gateway must be a Linux machine and doesn't necessarily need an IP Address

Raw Packets

• When a packet comes in, verify whether or not it is a brunet edge packet.
• If it is not and it is an IPOP packet, forward over the overlay
• If it is not and it is a generic Internet packet, forward to the proper gateway
• If it is an edge data, forward to the local gateway via ethernet header tweaking and IP over IP
• Implementation:
• Windows - Not possible without writing drivers.
• Linux - Could bridge a second tap device to the default network device (i.e. eth0) and rewrite Ethernet packet as necessary

Managing System

• Crawl, MapReduce Crawl, inconsistency investigator
• Analyzation result so far
  • MapReduce crawler result shows much better performance than sequential crawler.
  • If maximum map time is longer than routing time, task completion time depends on maximum map time
  • If maximum map time is negligible comparing to routing time, task completion time may depend on routing time.
• Future work plan
  • Handling lagging map task - related to rpc timeout
    • Wait forever
    • Ignore the map task result and return available reduce result
  • Measuring routing time for task distribution - To notify majority of time consumption for the map reduce task
• I implemented consistency checker using two methods
  • Method 1
    • Map function returns its own neighbor information.
    • Reduce function concatenates each Map function's neighbor information. Then, the reduce function calculates consistecny value using the neighbor information. If an entry is not needed for future calculation, reduce task removes the entry from the neighbor information.
  • Method 2
    • Map function gets its left1 and left2's neighbor information using RPC. Using the information, Map calculates its consistency value and returns the consistency value.
    • Reduce function adds returned consistecy value.
  • Comparasion between Method 1 and Method 2
    • In Method 1, each Map function needs only one rpc call to get its own neighbor information. Otherwise, in Method2, each Map function needs two rpc call to get its left1 and left2's neighbor information.
    • Method 1 needs more data transfer than Method 2 in the process of Map and Reduce.(neighbor information concatenation VS consistency value)
• I am focusing on better consistency calculation method.
  • Method 1 : Ask left2 neighbor's right neighbor. If the right neighbor is same as my left neighbor, hit occurs. Otherwise miss occurs.
    • In connection A-B-C-D, which D and B do not recognize C, but C recognizes B and D. D asks its left2(A) about its right neighbor(B). B is D's left neighbor, so hit occurs. C asks left2(A) its right neighbor(B). B is C's left neighbor, so hit occurs.
  • Method 2 : Ask left1 neighbor's right neighbor. If the right neighbor address is same as its own address, hit occurs.
  • Method 3 : Combining method 1 and 2
  • Finally I decided to use left1 and left 2 neighbor check. The way is as follows
    • Node A checks if its left neighbor's right neighbor address is same as its own address. If it were same, left1_consistency value increases by 1. It also checks its left2 neighbor's right2 neighbor adrress. If it were same, left2_consistency value increases by 1. We can get consistency value (left1_consistency + left2_consistency) / (number of nodes * 2)
• Way to organize efficient broadcast tree
  • broadcasting tree, which starts broadcast from left neighbor can miss some nodes in case of node inconsistency.
  • I am reviewing code, which starts broadcasting from left and right.
  • Comapring unidirection broadcasting and bi-directional broadcasting
    • It seems that there is no efficiency gain. This topic is out of scope for this paper. This will be future work.
• I am focusing on rpc error handling while getting neighbor information.

Setting up a Linux Gateway

The Server

The Server is quite easy to setup, simply add a masquerading rule as follows:

echo 1 > /proc/sys/net/ipv4.ip_forward
iptables -t nat -A POSTROUTING -s 10.8.0.0/24 -o eth0 -j MASQUERADE

This command assumes that the VPN subnet is 10.8.0.0/24 and that the local ethernet interface is eth0.

The Client

If we want to have the client redirect its default gateway through the tunnel, we simply need to redirect the gateway:

1. A static route to the other tunnel partner is created
2. The old default gateway is deleted
3. A new entry for the default gateway is created (pointing to the IP address of the other tunnel endpoint).

In Linux this can be done by:

route add tunnel_partner gw defeault_gateway
route del default
route add default gw tunnel_partner