mindshare

Thesis Finished!

Thursday, Mar 10, 11:23 PM

Last week Friday Florida Tech accepted my Thesis. You can currently read all 101 gory pages here. I'm pretty over the moon with being done with my degree and am taking a week or so to relax. At least one of the professors that reviewed the work was excited about its applications and wanted to know when it could be used in a real life classes. There is also the possibility that the crypto & security work may be taken up by a senior projects group.

Right now I need to do several things before development can continue. On the organisational side I want to change 3 things:

• Get the source code hosted in a Subversion (SVN) repository. SVN supports renames correctly for directories which is critical when you go to rename packages in Java, something I have no fear or compulsions about doing. We may eventually host downloads from Sourceforge but I have no interest in using CVS as the repository.
• Get a MediaWiki up and running to host all development discussion, like how they do for Firefox.
• Set up bug tracking with Bugzilla.
• Choose a Licence for the code and add it to all source files.

For an open source project those are the best in class tools. The SVN may be problematic because it requires special installation & configuration. The MediaWiki and Bugzilla should run from my current web host.

The other stuff is more about what direction the project should take. Particularly with regards to networking technology and JXTA. I got a profiler out and I am not too happy with what I see. Even small JXTA apps like MyJxta have a 50MB footprint (comparable to Limewire or Azureus) and they randomly chew up the CPU even when idle. JXTA also has issues with the speed of its DHT searches, aka the Discovery Service, which made it useless for presence support. It also takes a long time for each node to startup  and join groups. Interestingly JXTA is not I/O bound as one might expect, being a networking technology and all, actually its XML processing boutd, spending about 67% of its time munging XML documents. You get lots of cool stuff like NAT traversal 'for free' with JXTA, the question for me it just how free is it for my particular application?

Comments (0)

Logging works

Sunday, Jan 30, 11:23 PM

Sometimes I surprise even myself! And it only took 3 hours.

Comments (0)

JNLP & Logging

Sunday, Jan 30, 7:47 PM

I took a look at JNLP. I have made some choices about loading resources that wont allow mindshare to work as a JNLP app until they are fixed. Basically this was a testing convenience. I can set the working directory on each instance to be different and everything will be stored in that directory. This is great when you are running two or three peers on the same machine as a test. JNLP puts things in strange places and renames resources. The 'right' thing to do is to store everything in the users home directory. This would only allow one peer per user to run. The right solution it to allow the app to take command line parameters to put it into testing mode. This would probably take a day or more to fix so I wont get JNLP working any time soon.

I switched to Log4J from Java's built in logging API. I am very happy with the switch but the log window in the app has broken. What I really want is a coloured log output in the log window. Something like this:

 This would make debugging at any location just a bit easier. The output would be in HTML of course and you could save the logs. Trouble is there appears to be no off the shelf solution for hooking up a log appender to a JTextPane. i wrote the Layout in about 10 minutes though. I may take another crack at this tonight because I really like the idea of being able to read the logs anywhere.

Comments (0)

Changelog

Thursday, Jan 27, 5:29 AM

Not a very productive day but its the first real work i have done in about a week.

New:

• Work on BEEP connections between peers.
• Peers decide who is the 'client' and the 'server' by comparing the URI's of their respective JXTA PeerID's. The lesser PeerID is the client and thus responsible for making the connection.

Refactoring:

• All the old logging code is now replaced with Log4J.

Bugs:

• Fixed a subtle bug in Presence that happened when peers went offline and came back rapidly. The per record would be replaced and a timer task would remove a different but 'equal()' record.
• Fixed a synchronization bug in Presence shutdown where concurrent modification could occur in the peer list.
• Found a subtle bug in the buddy list display. If the list is interacted before all peers appear online it will not display new peers. Most curious.
• Also the buddy list is not being saved as it was before. This is actually a cool feature at this phase of testing so it may be made optional with the default being true.

Comments (0)

MediaWiki

Friday, Jan 21, 5:53 AM

MediaWiki MediaWiki looks cool.I'm a big fan of Wiki's in general and I want to use one here. I like the idea of using a Wiki for things like a manual, quick start guide etc. MediaWiki could handle this blog by the looks of it. JXTA has a wiki, however the primary resource for JXTA information and discussion is the dev mailing list. The problem with mailing lists and forums in general is information loss. Users must be clever enough to search for the right information. Wiki's encourage knowledge hoarding. Wiki pages snowball to cover all knowledge of that area. The Wiki's problem is page rot. You have to maintain and refactor the pages as things change or they become obsolete and misguiding. Forums and mailing lists are best for discussion transient topics where information loss is inconsequential.

Overall I like the Wiki over the mailing list or a forum so installing one is on my todo list probably in late February.

Comments (0)

Unique Unique ID's

Wednesday, Jan 5, 3:48 PM

Mindshare being the unique thing that it is needs a unique ID format. Each group member generates their own member ID. These ID's need to be unique, which is the usual requirement. Because the number of ID's create s small and scoped within a group the size of the ID's does not need to be that large. The 128 bit UUID format is probably overkill and Java cant generate true UUID's anyway because the JVM lacks access to several components of that format (MAC address and a high resolution timer). My initial response to this requirement was to Base64 encode some random bits.

The other requirements are more subtle. Mindshare save everyones files to disk and cares about who owns those files, something no other P2P program cares about. The user ID may be called upon to disambiguate files in cases where two files have the same name or to store each users files in a separate directory. This puts severe restrictions on the characters that can be used as ID's. For starters using Base64 encoding cant be used for file/directory names because it includes the '/' character which is either illegal or delineates directories on Unix platforms and in URI's.

Then there are the case-preserving, case-insensitive filesystems of the two popular desktop OS's. Base64 is case sensitive so raw Base64 cant be reliable used because two different Base64 strings might be the same thing as a file name.

Finally Base64 uses the '+' and '='characters. The = is easy to avoid by packing the base 64 such that no filler is needed. The '+' character falls into the punctuation group and is thusly not suitable for use in the authority section of a URI.

So for now Mindshare ID's will be 15 bytes of secure random data, encoded with Base64, resulting in a 20 byte ID. All capitol letters will be replaced with their common equivalent (id.toLowerCase()). The characters '+' and '/' will be replaced with the more URI friendly '-' and '_' respectively. There is still the possibility of collision in this scheme by virtue of the loss of capitalisation. I consider this to be unlikely or at least as nebulous as the possibility of generating identical random bits because Java doesn't have access to accurate spatial or chronological information. At some future time an encoder can be built that encodes raw bytes in the alphabet [a-z][0-9][-_] without changes to the protocol.

Basically this is a post to say I have thought about this issue but I am too busy/lazy to do the right thing at this point in time. This gets put off for 0.2 when we do crypto & security.

Comments (0)

Current Work

Monday, Jan 3, 2:15 AM

My thesis is slated to be handed in sometime in mid February. This puts a closed time-frame on work remaining before the alpha release when real people (users) will get to squeeze the software in their greedy little hands :-).

Give the time constraints the main priority is to get tree and file swapping implemented. Events have, however, conspired to make this not so easy. I lost some important code (formatted it into oblivion) that was not too trivial to write. To save time I am cutting XML as the back end data format. Right now much of the core already works on XML. The alternate encoding will be BEncoding much less stressful to work with. It solves many problems associated with sending binary hashes inside XML that cant contain binary characters. Right now I want to strip out all the XML code and replace it with BEncoded messages instead. The work needed to do this may be too much to be worth while for the alpha.

Presence support was rewritten to be more general. Integrating this back into the Mindshare client is a bit of a pain but the result should be a more responsive presence service.

All point to point links will use BEEP for transport. Before the code loss I had BEEP working over a JxtaSocket quite well. This uncovered a bug with closing JxtaSockets and BidiPipes that is now fixed. JxtaSocket still does not support keepalives so the protocol will include a keepalive packet. In the future a second BEEP channel could be used for one to one chat.

Heading up to the release I will try to post more often so check back.

Comments (0)

Website Swizzle

Thursday, Jul 22, 4:03 AM

I switched the index page to be an about page instead. This way people can learn about the project without being subjected to my rambling. Oh and check out the new web button:

Comments (0)

Screenshot

Monday, Jul 5, 11:25 PM

Here is a screenshot of build 260. It shows files being shared in a group and several users (just pretend they are all online). The user is about to give one of his files to another user. There are still a few GUI elements that need work, but mostly its finished. I've started working on the Thesis paper and in parallel ill be working on point to point pipes for file transfer. Enjoy the screenshot.

Comments (0)

I found the Presence Service Bug

Friday, Jun 11, 2:46 PM

I found the Presence Service bug! It only took 3 days. I spent the first 1/2 day exercising my bug hunting powers in earnest. All this revealed was that the bug could be anywhere and it would be wise to eliminate some of its hiding places.

First I rewrote the Buddy List view and model so that it used the domain classes directly. This caused me to rethink and refactor some things in the domain, particularly consolidating the source of all Presence events into the MindshareGroup class. Its nice to have events that start at the actual source of the event, Its also nice to have them aggregated at one location so listeners don't have to register with 4 different object.

After all this work the bug was still manifesting itself. I should have had more confidence in my existing display code. Once the display and network components were eliminated it was easy to find the bug in the controller. This prompted more refactoring to consolidate controller function into the Member Registry to avoid this happening again.

As a side bonus the groups now have a nice icon in the buddy list. I also learned allot about TreeModels and TreeModelEvents, things I will need for the group share work. Also the Buddy List display should be easier to modify when the back end is inevitably changed. Not a total wast of time but still no forward progress.

I've posted a new version with the changes.

Comments (0)