Advancing P2P technology

[Sepp Hasslberger]

This is a personal message of someone interested in advancing the technology of generalized p2p interaction on the net. You can contact Adam by email. It's adamsobieski @hotmail dot com, but perhaps it would be better to have the conversation here, so others can listen in...

Here's the message:

P2P Foundation,

Hello. My name is Adam Sobieski. I am a technologist who makes various software including artificial intelligence, distributed computing and P2P approaches to distributed computing. Lately, some of my focus has been upon e-learning applications, broadening e-learning capabilities and technologies. Additionally, an interest of mine is increasing the ease of multimedia use and publishing by end-users, and increasing multimedia to include interactive video. We can imagine audio and video blogs, as well as forums and interactive spaces.

I am interested in encouraging the creation of and search and retrieval of blogs and forums in intuitive ways, encouraging the expression of opinion by end-users, encouraging connections between people, content discovery, socialization, participation in civil society without motivated actors in their midst, and otherwise empowering end-users through technology that facilitates their expression of their reactions to the world around them as well as the expression of their desires with regard to their world. I view peer-to-peer networking as the logical technology upon which to construct next-generation blogs and forums.

I would also like developers to be aware of all the P2P libraries and utilities that exist. I would like to be more participative somehow in providing resources to developers to create new solutions on the P2P area.

I would also like to encourage somehow academic rapid prototyping of new research artifacts atop existing applications, clients, browsers, various objects where a branch or modification of an open source project can rapidly and effectively communicate an idea in a participative way with the scientific community in the form of the artifact and related publications. This, in my opinion, benefits all of the involved scientists and facilitates additional communication between scientists through constructed variations of artifacts exhibiting prototypical research functionality.

Also, we can imagine a distribution network for RSS about new software, versions and modules in the P2P area. I feel that every developer has an interest in the means of connecting with interested end-users, additionally many software have web-based means of apprising users to the release of versions. I would like to see this integrated into a desktop taskbar notification area as opposed to each application running a separate process to poll a web service to periodically check if an application popup is required. This an integration methodology for new components and versionings can be of use. In particular as live distributed applications have more complex versioning scenarios. I view "distributed versioning" as a pragmatic complexity that can be alleviated with a shared network resource providing a service to end-users as well as allowing bolder advancements. The means by which distributed applications version in a distributed manner and the means by which end-users are apprised of new components and modules of interest to them. I think that we developers would like that, that with an integrated widget, possibly on the taskbar. Desktop integrated software arrival and versioning notification through a decentralized and distributed resource or network combined with user settings and ergonomics.

I wanted to introduce myself, say hello and indicate the above idea for a distributed resource. I'm composing also some ideas regarding implementations of completely decentralized distributed peer-to-peer blogging and forum systems. If anybody knows of existing research in those areas I'd be interested in reading the science in that area.

[Sepp Hasslberger]

In an email, Adam said:

How do we archive these things for ourselves and for future historians? We can design this into distributed systems to provide information search and retrival for recent data, with distributed computation compressing that into medium term memory (large compressed files available for segmented download) then those automatically get sent to archivists, potentially university history departments around the world.

and

Specific to archiving of and by distributed systems, my initial thoughts on the implementation related details are: multicore compression, distributed compression, segmented downloading of large archive files, search and retrieval of them and potentially of items in those files, and routing these over time to repositories such as those at history departments.

My view is that archiving a conversation that travels on normally short distance lines and that is in the context of p2p type software, will not be a straightforward task like archiving web pages in the internet archive's wayback machine.

Here is what I said:

I believe we should distinguish between a general archiving of the data that have been made available on the net and the archiving that will eventually have to be thought out for all the conversations generated on a future p2p type parallel or sub net.

For the www itself, yes, it would be great to have, for instance, all paper and magazine morgue files available, not just to historians but as a generally searchable data base that provides a record of all print publications back to as far as we can possibly make it. In a wide sense, this would include book initiatives such as Google's scanning of orphan and out of print works and making those available. This alone would be a huge and very worthwhile project.

The Internet Archive seems to be doing a good job of archiving the conversations that happen on the net today, articles on websites and discussions. Social networking sites have a lot of valuable conversation going on, and I am not so sure this is being included in what's currently archived. After all, here we have a huge number of communications, many of them frivolous but also many serious ones, the totality of which is not visible anywhere except on facebook's own servers. Each user only gets what pertains to their friends, and only a selection of what is considered more interesting, but that user-specific rendering then vanishes into a long tail of conversations that are no longer easily accessible much less searchable.

On the p2p side, a similar problem will present itself, but with the caveat that we won't even have a central server to refer back to, or to spider for the record of what is happening. As you say, distributed spidering will probably be needed, but what about the locale of storage? History departments will be interested in having access to that data, but they will not be necessarily the best custodians for the raw stuff.

Tentatively, I see a way of handling this by leveraging the computing and storage power our own computers have. Take email conversations, for instance. People have a lot of data in those conversations, including attachments, which are being kept by email clients on some of our computers. Then each one of us has some sort of storage system for interesting files on their own computers. With a p2p direct net developing, there is going to be more and more of that type of data sitting on our computers, partially replicated on other computers, but nowhere collected into a coherent story or centrally archived for later retrieval and historical interest study.

I am thinking people should be made aware of this and a discussion started, with the aim of preserving the data on people's hard disks for historical evaluation. I would have never said such a thing a decade ago, but ... our notion of privacy is changing and, after some time has passed, all that data does seem much less personal. For instance, we are happy to have the correspondence archives of the scientists of the past, which at the time they might not have opened for public access. Only with their passing have the archives become accessible to researchers. Something very similar should probably be envisioned for people's hard drives. Perhaps a "donate your old drive" or "donate your backup to historical archives" type campaign could help in this, or it could be done by spidering and archiving.

My point: We are moving away from a server based model (where archiving was fairly straightforward) towards a more user-based model, where everyone keeps their own data of interactions (conversations) with others and associated connected media files. We will therefore need to re-think the way we archive those conversations for future generations. Technology advances will help in unexpected ways, but perhaps the basic model should be in place before p2p really starts taking off.