Backing up the internet in a P2P 'cloud'

[Sepp Hasslberger]

With optical cables being cut in the Middle East impeding internet traffic in some geographical areas and a recent court action resulting in WikiLeaks being taken off line, it becomes clear that access to the net may not be as assured as we tend to think.

While the basic architecture of the net does protect to some degree against these dangers, I believe we might profit from developing a way to "back up the internet" so that, even if there are major disruptions, we still have a workable means of communication, data storage and exchange of ideas.

My dream is a p2p application that uses some of the free hard disk storage space on our personal computers to redundantly back up the net and allow work to continue more or less seamlessly in the event of a major catastrophe.

Would such a thing be doable? What do you think?

Is anyone already working on this?

[Martien van Steenbergen]

Thank you Eggy1943.

In fact, I scribbled up the Wizard Rabbit Treasurer after having read a number of papers on mobile network protocols, roaming en reputation. As well as incorporating all the things I have learned in my 12 1/12th years at Sun Microsystems, where they have the mantra The Network Is The Computer™ since 1987... And where NFS, Java, Jini, JXTA and ZFS originated. I feel lucky to have been there. Longing to see all this become a reality soon. Our Aarde is in metamorphosis as we speak. Wonder what will emerge.

Have a nice weekend.

[Paul B. Hartzog]

In regards to your question on whether anyone is working on
a "Peernet" future infrastructure on p2pfoundation:

I have been working on this for years,
with Keith Smith and Matt Berlin.

Keith and I created an architecture,
which Matt and I have attempted to find funding for.
We have had various names from "EcOS"
to "Knowledge Commons" but I like your "Peernet."

I originally presented the idea to Howard Rheingold,
and Kathi Vian (IFTF) and Andrea Saveri (IFTF)
in 2004, and then Kathi flew me to her office/home
in Woodstock NY for a week of working on it together.
Afterwards, we pitched it to Sun, and a few other big
vendors, but nothing has come of it yet.

Peernet is effectively an alternate economy,
and this is precisely one of the things that our team has
discussed, which also led to my article about
The Future of Money (which Adam Arviddson recently cited).

I am VERY interested in pursuing
this (as Sam Rose and I have been discussing).

Perhaps we should line up a skype call or find a way
to really talk about this in some detail: architecture, etc.

[Sepp Hasslberger]

It would be really great to see some progress on this, and to have a link to a paper or page of what you're working on that people can check out.

Of course the funding question is a difficult one.

It seems that there are various strands of ideas that have been conceived, but not yet implemented either for lack of funding or for lack of the 'go-getting enthusiasm' needed to transform them into actual working projects.

Just off the top of my head there are

Martien van Steenbergen in the Netherlands, long time collaborator at SUN, with his Armillaria project that doesn't seem to have attracted the support it deserves

Simon Edhouse, who says he has been working for years on a solution that he isn't quite ready to discuss in public yet and

Sam Rose and Marcin Jakubowski working on their Point to Peer project as pointed out by AGNUcius.

Others know of some more??

Maybe we should increase communication and interchange between the various solutions that are being hatched to cross-fertilize?

;-))

Sepp

[Michel Bauwens]

To Martien's query about what is being done.

Are you familiar with those names:

# For the most fully distributed Peer to Peer Computer Networks at present, please check out Tribler ;Peerple ; Wipeer
# Research into more fully distributed P2P systems for the future: Chord, CX Project, Farsite, Globe Project, Oceanstore, Pastry

I believe Olivier Auber, Gwendal Simon, and Anh Tua would be able to answer your technical query,

Michel

[Dmytri Kleiner]

Hello, AFAIK, this has been an area of research for quite some time, i.e. the pStore project, Mnet, etc, however the problem, here, as with P2P in general, is that P2P is incompatible with the mediation required by Capitalism to capture value by controlling circulation, thus P2P projects tend to be unable to provide for their economic requirments.

As an all too typical example, Mnet's lead developer quit the project as a direct result of economic reality:

http://zgp.org/pipermail/p2p-hackers/2005-March/002483.html

The project promptly died, despite the quite significant progress already made.

While there have been some exceptions, the overall trend is for P2P systems to die and for centralized
systems, drivin by Capitalist finance, to dominate.

Cheers.

[Dmytri Kleiner]

Hi Again.

IMO, the most promissing projects currently in this space, are tahoe and flud.

"flŭd backs up your files to multiple remote locations using techniques that can produce the most resilient and secure backup system yet devised, for free. flŭd is experimental software."

http://flud.org/

Flud also maintains list of simular systems:

http://www.flud.org/wiki/SimilarSystems

Tahoe is a project of allmydata.com, a company that provides windows and mac only commercial storage system using a similar peer system as well as a more traditional "webdisk" approach. I'm not sure if the commercial product is based on tahoe, or if it is comepletely seperate.

Tahoe seems promising because it comes from the MojoNation/mNet community, which has significant experience in this space.

http://allmydata.org/trac/tahoe

However, I have not tried any of these systems I would be interested in hearing about any opinions or experiences regarding the stuff on the flud list, or others.

Cheers.

[Sepp Hasslberger]

Hi Dimytri,

thanks for all these links to systems that allow a backup of personal data with peers on the internet.

My idea was really the other way around, I was looking for a way to back up the internet, or a sizeable portion of it, on a p2p network of our own.

The reason for that would be to increase survivability in case of widespread disruption of the current internet infrastructure. The picture I was imagining was a holographic record of what's on the internet, redundantly stored on the free disk space of numerous personal computers, which could serve to reconstruct at least part of the data on the net after a catastrophic occurrence.

Perhaps it is like someone commented on another string of this discussion, that I am imagining a job for p2p that is both too ambitious and not really necessary.

Anyway, I really just brought this up for the sake of having a look at the possibility. And who knows, perhaps with advances in both storage capacity at our disposal - we have already evolved from hard disks of a few megabytes to now measure space in hundreds of gigas per perhaps soon in terabytes - and with smarter storage algorythms, in the future this might become doable.

Sepp

[Josef Davies-Coates]

Hi Sepp,

Whilst your enquiry was indeed about backing up the Internet, surely data is data and a the technology required to provide p2p back up for copies of web pages or personal files (some of which may well be web pages too) is basically the same?

Both are about p2p back up of data. What that data is doesn't make a lot of difference to solving the problem and so Dmytri's links are not really "the other way around". :)

Smiles,

Josef.

[Sepp Hasslberger]

Yes that's true Josef,

the difference is only in the magnitude of the task :-))

Sepp

[Michel Bauwens]

From Jaap van Till:

Nice idea but I think you must distinguish between (a) the worldwide cable and network infrastructure
( comparable to the road infra) and (b) the databases, application system and presentation devices (comparable to cars) of the network of networks. If a. is disrupted there not very much you can do anymore for a long time with local or off-line stuff (b). In the comparison: you can maybe still drive around your garden and in the neigborhood, but it is rather hard to backup the interstate roads yourself. Problem i that not very many people have even a vague
idea what and where (a) is. It is invisible behind their laptop screen. And we are accustomed that it works, ok slowly sometimes.

For (a) and its vulnerabilty there is a large body of knowledge and science available, from for instance Barabashi,
(sorry I can have no time to verify the correct spelling, he is an usa professor with an Hungarian name). Eric Luiijf and also I have done further unpublished research on SPOFs (single points of failure) and vulnerabilties.
I even published a "conjecture" that if things are implemented more distributed (and therefore the whole is less vulnerable) some things move to more central and unduplicated places. A kind of law of conservation of vulnerabily.

The main point is that although the mesh structure of Internet infra is very resilient and works in parallel and with very much redundant components, there are many places where if disrupted (if you know where to attack) large parts of the internet will go down. I am not allowed, nor will I volunteer, to give examples to tell you where those SPOFS are.
Sometimes these arose because of cost savings by the managers who let the networks be built.
The only way to find them and do something about them is in my opinion testing and excersising under
controlled circumstances, and of course a learning process from "incidents" like the FAA does after planecrashes. One of the problems is that carriers and operators say they have everything under control and that
their network management rooms are able to face any disturbance or outage. But what when a number of unlikely things happen at the same time? So they tell governments that that nothing bad can happen. This is similar to the airlines saying that no planes can crash. Again the answer is not yes vs no, but how we can install a mechanism to learn from unexpected incidents. Need for a FNA (Federal Network Authority) maybe ?

By the way the gossip about the undersea cable disruptions now is that they may be caused by organised sabotage. But by whom ?? Is it a test or exercise from our friends, in line with my described line of thinking ?

Jaap van Till

[Sepp Hasslberger]

Hi Michel, Jaap,

as you correctly point out, my original post here actually mixed two distinct problem areas into one.

One area is preserving data by a distributed backup of what's important and by providing a mechanism of re-constructing the data in case major parts of the server space should disappear.

The other area is backing up the physical network by creating a more distributed architecture and redundant modes of interconnection.

Thanks for an interesting discussion of the underlying network infrastructure and its inherent vulnerabilities.

The Japanese have done an interesting thing after the recent cable cuts: They put up a research satellite that can provide connectivity in Japan and between Japan and other East Asian countries ... they say they're not offering this for public use, just doing some research :-))

[Sepp Hasslberger]

Back up the internet on the moon

says Xenophilia in this post in a (half) joking way, reporting on a proposed earth/moon optical data link.

Great idea. Off site (off world) backup of the Internet and all knowledge we have thus far accumulated. Put a copy of this blog up there too. Upload everything we’ve got to some huge hard drives on the moon. Put another copy on Mars and continue on from there...