From Grid-Appliance Wiki
IPOP (IP over P2P) is an open-source research/development project that provides a decentralized, distributed overlay virtual network. It allows users to easily deploy VPNs across multiple domains, and is used in applications including virtual private clusters for cloud/grid computing, and private friend-to-friend communication for social networking users. IPOP is written in C#, runs on .NET and mono, and is licensed under the MIT open-source license. Key features of IPOP include:
- Low configuration overhead: VPNs can be configured through Web and social network interfaces
- Self-optimizing connections: the virtual network autonomously detects communication between endpoints and create direct connections
- Dependable P2P overlay: the topology and routing of the underlying P2P include various fault-tolerance features to handle physical network connectivity constraints
- Support for computers behind NATs and Firewalls: IPOP supports a STUN-like decentralized NAT traversal scheme for cone NATs, as well as autonomous selection of relay nodes for symmetric NATs.
IPOP is a virtual network that allows computers distributed across the world to communicate using the IP protocol over virtual network interfaces. It does so by creating a virtual network device on each computer joining an IPOP virtual network, encapsulating IP packets in overlay messages, and using a scalable and resilient P2P fabric for routing messages. Unlike many P2P systems that focus on data storage/lookup and file sharing, IPOP is tailored and optimized for handling virtual IP communication. IPOP abstracts away many of the complexities of the underlying physical Internet; in particular, IPOP provides features that allow two machines constrained by different NATs (Network Address Translation devices) to communicate with each other through traversal techniques as well as through the P2P overlay (akin to triangular routing) without configuring any routing rules. Two of its main use cases are:
- Cloud computing: IPOP is the basis for GroupVPN, a VPN allowing users to create clusters of virtual machines that, while physically distributed across multiple domains, appear as a logical virtual cluster where nodes are bound to a private address space subnet.
- Social networking: IPOP is the basis for SocialVPN, a VPN that allows users of social networking infrastructures to communicate privately and directly to each other through self-configuring VPN links
IPOP is an active project currently in use by other projects, including:
- SocialVPN, a self-configuring virtual private network connecting online social network users
- Grid Appliance, a self-configuring Virtual Machine appliance that is used to create ad-hoc pools of computer resources both within a local-area and across wide-area networks to execute high-throughput, long-running jobs.
- Archer, a distributed virtual private cluster for computer architecture research and education
- FutureGrid, an experimental testbed for research and education on Grid and cloud computing
- ConPaaS: IPOP is being integrated into ConPaaS, an integrated cloud environment for elastic cloud applications
- PRAGMA: IPOP is being integrated into the Pacific Rim Applications and Grid Middleware Assembly
IPOP integrates with existing kernel-level tap virtual network devices for packet capture/injection, and implements a user-level P2P VN router. The user-level IPOP router builds upon the Brunet P2P library, which provides the core services of routing, object storage/lookup, and overlay connection management supporting multiple transports (TCP, UDP, and tunnels) and NAT traversal. Public IPOP bootstrapping overlays run continuously on the PlanetLab infrastructure.
IPOP is currently undergoing a major design overhaul, with major goals of 1) incorporating standards and open-source implementations (including STUN, TURN, XMPP, and libjingle), 2) modularizing the design to support multiple overlay topologies, and 3) supporting Android mobile devices. The tentative release date for the next release of IPOP based on this new design (and Web page overhaul) is Jan 2014. For a preview of the new codebase, please check the new IPOP project GitHub repository.
Online Course and Hands-on tutorials
If you'd like to learn more about IPOP, this free, open online course has 12 short video lectures describing the scope, architecture, implementation, and uses of IPOP.
Want to test-drive IPOP? The following tutorials are quick and easy, and you'll get a sense of the basic capabilities of the system. All you need is a couple of virtual machines.
- June 2013: IPOP MOOC module overviewing the technology and its applications is now available!
- June 2013: IPOP is now part of ConPaaS 1.2.0, thanks to the great work of the ConPaaS team!
- June 2013: Renato Figueiredo presented at the first ConPaaS workshop in Amsterdam - the presentations give a nice overview of the ConPaaS platform-as-a-service system, and how IPOP fits in its context to provide virtual network across multiple clouds.
- May 2013: Renato Figueiredo presented an invited seminar on IPOP virtual networks and applications, at the System and Network Engineering (SNE) research group from Universiteit van Amsterdam, the Netherlands.
- May 2013: Renato Figueiredo presented at the Panel on Clouds at CCGrid 2013.
- Mar 2013: Pierre St. Juste presented an IPOP poster and demo at the PRAGMA-24 Workshop in Bangkok, Thailand
- Mar 2013: Renato Figueiredo presented a Webinar on IPOP and applications to Milibo members.
- Feb 2013: Renato Figueiredo presented a keynote seminar on IPOP at the ASCI DAS Workshop in Delft, the Netherlands
- Feb 2013: A demo of ConPaaS, which included the use of an IPOP virtual network, was given by Emanuele Rocca during the 10th Contrail General Technical Meeting in Pisa, Italy
- Dec 2012: Renato Figueiredo presented a keynote seminar on IPOP at the Grid'5000 2012 school in Nantes, France
- Jun 2012: Renato Figueiredo presented a seminar on network virtualization at the Contrail 2012 Summer School in Almere, the Netherlands
Security: IPsec, GroupVPN, SocialVPN
IPOP is the baseline overlay virtual network software that provides virtual network connectivity among peers. Privacy, integrity and authentication in IPOP-based VPNs can be accomplished in several ways:
- IPsec: Because IPOP tunnels IP messages, including IPsec packets, VPNs based on standard IPsec technology are possible. Users can employ IPsec stacks to setup VPNs. The configuration of IPsec, however, is currently not automated nor part of the IPOP package; it requires users to configure their own IPsec stacks, such as Raccoon or StrongSwan. This page provides an example of the configuration of Racoon in a setup using X.509 host certificates signed by a trusted CA to create a VPN.
- GroupVPN uses IPOP's own security layer and provides users the ability to create their own private networks using group environments to secure and manage the system. It is quite similar to Hamachi in essence; what differentiates the approach from Hamachi's is the focus on decentralization. Users can create their own groups through our website.
- SocialVPN uses IPOP's own security layer to allow users to seamlessly create their own private networks connecting them to friends. SocialVPN uses online social networks to discover peers and exchange public key certificates (by connecting to a XMPP server, such as GoogleTalk, or Jabber) and IPOP to tunnel virtual network messages.
Differences among security frameworks:
- IPsec allows the use of widely-deployed security software integrated with O/S kernels (e.g. Racoon/Linux), while GroupVPN and SocialVPN use a security layer implemented within the IPOP codebase. The main disadvantages of IPsec are complexity in the installation and configuration of the IPsec stack; the security layer of GroupVPN and SocialVPN is user-level and runs within IPOP.
- SocialVPN allows each individual to determine who joins the VPN and who doesn't, whereas the GroupVPN allows a group owner to determine who joins the VPN; users of the VPN are connected in an all-to-all fashion
- SocialVPN provides each user their own view of the network, and to support limited availability of IPv4 address spaces, it produces a NATed environment for each user. The GroupVPN places all users into the same IP subnet.
Documentation and code
- First-time user's tutorial: Hands-on tutorial on how to run IPOP on Debian/Ubuntu virtual machines
- Deploying your own IPOP bootstrap: Hands-on tutorial on how to run a private IPOP bootstrapping overlay on Debian/Ubuntu virtual machines
- ACIS P2P video channel with demonstrations and tutorials.
- Presentation slides and an open Online Course with a technical introduction to IPOP and applications.
- Brunet Issue Tracker
Software: Current Release
A Debian package for IPOP is maintained at grid-appliance.org; to install it:
echo "deb http://www.grid-appliance.org/files/packages/deb/ stable contrib" >> /etc/apt/sources.list wget http://www.grid-appliance.org/files/packages/deb/repo.key apt-key add repo.key apt-get update apt-get install ipop
Current Development Source Code
- ACISP2P Google Group: Discussion list mostly related to development of P2P/IPOP projects in our Lab.
- ACISP2PUSER Google Group: Discussion regarding to installation, configuration, administration, usage, and your own development with our projects.
- Kyungyong Lee
- Yonggang Liu
- Pierre St. Juste
- Jiangyan Xu
- Thom Dixon
- Ben Woodruff
- Dr. David Wolinsky
- Dr. P. Oscar Boykin
- Dr. Arijit Ganguly
- Abhishek Agrawal
- Dr. Girish Venkatasubramanian
- Han Zhao, Ze Yu, Shivam Tiwari, Xing Mao, Kyungyong Lee, David Wolinsky, Xiaolin Andy Li and Renato Figueiredo, "CloudBay: Enabling an Online Resource Market Place for Open Clouds", The 5th IEEE/ACM International Conference on Utility and Cloud Computing - UCC'2012
- Kyungyong Lee, David Wolinsky, Renato Figueiredo, PonD : Dynamic Creation of HTC Pool on Demand Using a Decentralized Resource Discovery System, Proceedings of the 21st International ACM Symposium on High-Performance Parallel and Distributed Computing (HPDC-2012)
- David I. Wolinsky, Panoat Chuchaisri, Kyungyong Lee, Renato Figueiredo, Experiences with Self-Organizing Decentralized Grids Using the Grid Appliance, International Journal of Cluster Computing - 2012
- David Wolinsky, Renato Figueiredo, "Experiences with Self-Organizing, Decentralized Grids Using the Grid Appliance", Proceedings of the 20th International ACM Symposium on High-Performance Parallel and Distributed Computing (HPDC-2011)
- David I Wolinsky, Pierre St. Juste, Oscar Boykin, and Renato Figueiredo, 'Addressing the P2P Bootstrap Problem for Small Overlay Networks', In the Proceedings of the 10th IEEE International Conference on Peer-to-Peer Computing 2010 (P2P'10), 08/2010. PDF.
- D. Wolinsky, K. Lee, P. O, Boykin, and R. Figueiredo, “On the Design of Autonomic, Decentralized VPNs”, In the 6th International Conference on Collaborative Computing (CollaborateCom 2010), October 2010
- David Isaac Wolinsky, Yonggang Liu, Pierre St. Juste, Girish Venkatasubramanian, Renato Figueiredo. 'On the Design of Scalable, Self-Configuring Virtual Networks'. In Proceedings of SuperComputing '09 (SC), 11/2009. PDF.
- David Wolinsky, Yonggang Liu, Renato Figueiredo. 'Towards a Uniform Self-Configuring Virtual Private Network for Workstations and Clusters in Grid Computing'. In Proceedings of Virtualization Technologies in Distributed Computing (VTDC). 06/2009. PDF.
- Arijit Ganguly, David Wolinsky, P. Oscar Boykin, Renato Figueiredo. 'Improving Peer Connectivity in Wide-area Overlays of Virtual Workstations'. In Proceedings of IEEE High Performance Distributed Computing (HPDC). Best paper award, 06/2008 PDF
- Arijit Ganguly, David Wolinsky, P. Oscar Boykin, Renato Figueiredo. 'Decentralized Dynamic Host Configuration in Wide-Area Overlay Networks of Virtual Workstations'. In Proceedings of Large-Scale and Volatile Desktop Grids (PCGrid), 03/2007 PDF
- Arijit Ganguly, Abhishek Agrawal, P. Oscar Boykin, Renato Figueiredo. 'WOW: Self-Organizing Wide Area Overlay Networks of Virtual Workstations". In Proceedings of the 15th IEEE International Symposium on High Performance Distributed Computing (HPDC), pages 30-41. Paris. PDF
- Arijit Ganguly, Abhishek Agrawal, P. Oscar Boykin and Renato Figueiredo. 'IP over P2P: Enabling Self-Configuring Virtual IP Networks for Grid Computing'. In Proceedings of the 20th IEEE International Parallel and Distributed Processing Symposium (IPDPS). Rhodes Island, Greece. PDF
Interested in contributing to IPOP? Let us know! Join our user's group, or send an email to the project faculty.