Friday, September 23, 2005

Creating a timeline analysis tool

I'm not a DBA, and I don't play one on TV. So when it comes to writing a timeline analysis tool, I'm not going to be able to do it myself. But here's what I propose...

I'm most comfortable on Windows platforms, and I know folks that prefer Linux, so that's cool. I know that there are people out there who are familiar with databases, and others that are good graphics programmers. What I'd like to see about doing is opening a project on SourceForge, and see if we can't get a decent start on developing a solution. Here's how I envision it going...and please keep in mind that this isn't the be-all-and-end-all, just my impressions:
  • Identify data sources (we've already gotten started with this, but we don't have to be restricted to Windows systems)
  • Identify data sets (data source/log normalization, etc.)
  • Identify a database format, using mySql (table definitions, etc.)
  • Identify and create a graphic component for presenting the data sets
So...here's what I think we need...and please feel free to weigh in here...we'll need someone to take responsibility for the database structure. I think I remember some good install instructions for mySql on Windows - if anyone knows of a resource that's not from the mySql site, please let me know. I do think that the database is pretty important...even if the actual graphic representation of the data never really comes to fruition, at least we'll have a good structure for a single structure to analyze the actual data.

As far as the graphics programming goes, I'm not really sure where to go with that one. Would it be better to go with some of the stuff that's already out there, or create something new with Java or some other cross-platform solution?

So...who's interested? If I get enough interest, I'll go ahead and see about creating a project on SourceForge. Like I said, I can provide various means of extracting the data. For example, most of my Perl scripts that parse raw binary files from Windows can be made cross-platform, and easily modified to dump the information into a database. I can also write the necessary scripts for ProDiscover. Providing solutions to pull data from a source and populate the database is something that will be on-going, I'm sure.

Now, there's no doubt in my mind that this sort of project will take a while...if it were easy, everyone would be doing it. I don't expect it to be done overnight. However, I would like to see it get done, b/c I do think that it would be extremely useful to a lot of people.

So...thoughts?

Addendum: I just did a search on SourceForge for "timeline" and there are several projects listed, but most don't have anything more than a simple page...no project files, nothing. There are some projects with files, but I'm not entirely sure that they'd be suitable.

However, there is something promising...check out the Timeline view of GenealogyJ. My concern would be the volume of data, but it does look like a good start.

Also, my hope is that this will be compartmentalized...meaning that the graphics component won't be database-dependant. That way, there can be several different interfaces for presenting the data, and they can develop over time.

1 comment:

Anonymous said...

Did you take a look at rrdtool? It has all the requirements you need. Database, graphing capability. All you need to do is to pump the data into the rrd database. Plenty of documentation around the web for rrdtool. Hope this helps. http://people.ee.ethz.ch/~oetiker/webtools/rrdtool/