Occam’s Razor

The Two Sides of Google

Google is one of these amazing companies that demonstrates how uninspiring and mediocre most businesses in the Information Technology (IT) field actually are. Unlike Microsoft, which claims to be innovative but largely is not, Google can truly claim the mantle. Google is a company with the power to inspire awe. Its search engine continues to be the cream of the crop. Yahoo and MSN will keep trying to best Google, but they will likely continue to play follow the leader. Yahoo Maps, for example, just recently released its Beta mapping application, which roughly compares with Google Maps. Google Maps, of course, has been using the magic of AJAX (Asynchronous Javascript and XML) for over a year to take map usability to a completely new level.

For software engineers like me, the speed with which Google churns out amazing new technologies takes my breath away. Its billions of dollars in ready capital certainly explains part of its success. With its passion for excellence and fearlessness taking big chances, Google simply soars high above the rest of the IT crowd. Mostly it hits the bullseye. Google Earth is just the most recent example of a technology that blew my socks off. It is a killer application, as every bit as revolutionary as the first web browser. We were still being wowed with Google Maps ease of use when Google threw us the Google Earth bombshell.

One of the more recent services introduced by Google is Google Video. It provides a new way to find and share video files. Google acts as the Internet’s ubiquitous high speed and fault tolerant video server. Given its enormous infrastructure, hosting and serving these large bandwidth intensive videos must not be much of a problem. The service even lets you know its most popular videos. Yet this is just one of a number of flashy services that Google provides, most at no cost. Let Google host your blog on Blogger. Centralize your email on the network with GMail. Find the lowest price online with Froogle. Search your computer as you would the Internet with Google Desktop Search. (It just happens to be a feature of the Google Desktop, a clever new application, which looks like a first attempt to break up Microsoft’s desktop monopoly.) Google even has pretensions in the Instant Messaging arena with its Talk client. Clearly, their ambition knows few bounds. While it occasionally bites off more than it is ready to chew (GMail being an obvious example) Google’s numbers of home runs outside the ballpark would make even Babe Ruth jealous.

Scott McNealy of Sun Microsystems was I believe the first to promote the idea in the 1990s that “the network is the computer”. While he was ridiculed at the time, Google has shown us that the network can be the computer. With an infrastructure like Google’s, what seemed impossible can unexpectedly become reality. While Microsoft spins its wheels trying to make its Windows product ever niftier, Google shows us that it is what you can do with a computer that makes it meaningful. Indeed, Google makes a compelling case that its services and infrastructure is the ubiquitous application layer of the future, if not the here and now. I am creating this entry using Microsoft Word, but I have already checked a half dozen facts online using Google’s search engine. I can use any word processor to create this entry. I cannot go just anywhere online to find out the information I need this rapidly. Google demonstrates it is not how pretty your screwdriver is that matters, but how well it helps you turn the screw. Therefore, we get its low-tech web pages, always with the pure white background, the simple text and its generous use of white space. It appears low tech but it is simple enough for a student in grade school to use effectively.

So I have plenty of cheers for Google today. I am especially glad it gave the U.S. Justice Department a Bronx cheer when the department recently requested a week’s worth of its web searches. The Justice Department wants the information to discover how the web is being used by pedophiles and those interested in child pornography. Unlike Yahoo and MSN, Google wisely said no. It valued the trust it has earned with its customers too much to let the Justice Department mine its information. Let us hope it continues to do so. Apparently, Google records the Internet Protocol (IP) address of every search query. Let us hope that if push comes to shove Google simply stops recording the IP addresses of all our search queries. For an administration already deeply in Big Brother land with its warrantless electronic eavesdropping, this is simply an opening salvo by the government to get its hands on our private business. If Yahoo and MSN care that little about my privacy, I will not be giving them my business.

In making a stand in America though, Google apparently is quite willing to compromise its principles to win business overseas. For also in the news this week were stories that Google will allow the Chinese government to censor its search engine content. Maybe I was naïve, but I really thought Google got it. However, apparently they will compromise their principles if it improves their shareholders’ bottom line. Perhaps as a result, Google shares went up 3.4 percent with the announcement.

Google must not understand its own unique power at this point in history. Many of us talk about the importance of human freedom, but few are in a position to do much to expand it. Google can. It is the 900-pound gorilla in the information search business. Rather than kowtow to China’s paranoid rulers it should have said no thanks. Yes, perhaps that might have kept Google out of the important Chinese market. Yet a powerful and uncensored internet search engine is a great beacon for those who believe in the power of ideas. The Google business plan surely was premised on its importance. Google is a trusted broker for finding uncensored information. It expands personal freedom and spreads enlightenment. Its reputation is at stake. Which is why Google should rethink doing business with China. Right now, its search engine is the largest force for the liberation of the human mind in the 21st century. Google can be both profitable and spread human enlightenment at the same time. It should tell China it does not need its business unless it guarantees that its citizens have the unfettered access to its search engine.

January 29th, 2006 at 08:47pm Posted by Mark | Politics 2006, Technology | no comments

The Id Unleashed at Craigslist’s Casual Encounters

(Warning: adult content. Reader discretion is advised.)

I am on a two-week holiday. I go back to work on January 9th. Like last year, I try to make my vacations at home count. Since my scenery is not going to change, I find it helpful to change the scenery in my mind. One way is to venture to places on the internet that time does not usually allow me to visit, such as Craigslist. There I go trolling for the unusual and the bizarre. I do not know what it is about Craigslist, but bizarre people seem to be drawn to it like a moths to a flame.

Because it was one of the internet’s first successful bulletin boards (and because its owner Craig Newmark has persisted in keeping it around for about a decade), Craigslist has proven surreally popular. It is achingly low tech and unpretentious. It seems to be inhabited by contributors who cannot bothered to spell check or use mixed case. Sadly, it is often full of spam. It is also a zone in cyberspace absent virtually any of the constraints that come with real life. As best I can tell, the only thing that you cannot get away with on Craiglist is child pornography. You can buy or sell pretty much anything including apparently illegal drugs (using code words like 420). It is one massively low-tech free speech zone. Except for the cost of an internet connection, it is truly free.

If you want a taste of Craigslist at its most bizarre, you simply must check out its many Casual Encounters sections. (There is one for each major city and state). But beware: Casual Encounters is an ultimate smoke and mirrors place in cyberspace where kinky and desperate libidos (generally male) play war with endless numbers of con artists and massively dysfunctional people. In Casual Encounters, the id emerges into the limelight and hold us in its garish gaze. Reality becomes surreal and the surreal morphs into the real. No one is quite who they appear to be. Their many kinky needs suggest posters are dealing with larger issues. Perhaps just by reading them from time to time I too am one of the fallen. (Just because I suspect you will be wondering, no, I have never met anyone from Craigslist, except my wife, who found some work on the side through Craigslist when she was otherwise unemployed.) On the other hand, maybe I am just a human being. Just as most drivers cannot help but slow down and gawk when we pass by an accident, those of us straying through Craiglist’s Casual Encounters find it something of a 24/7 rubberneck zone. It is not pretty. It is not poetic. It is typically crass and obscene. Nevertheless, it is hard not to look.

Most of the posters are guys, of course. It is clear their hormone levels are critically high. It is also clear that most haven’t a clue how on to woo a woman. Perhaps this characterization is unfair. Perhaps it is more accurate to say they have many very kinky needs that they cannot whisper to a woman they know. They can only find safety in expressing it to someone whom they do not know. Many seem incredibly desperate. Here is a sample of some of today’s postings from men looking for women in Craigslist’s Washington D.C. Casual Encounters area:

• A 26 year old guy has his webcam ready and is willing to show his engorged nine inch long sexual organ to any willing woman.
• A 24 year old guy woos women with, “You be Dairy Queen and I’ll be burger king. You treat me right; and I’ll do it your way”. (You can bet women are swooning over these lines.)
• A 25 year old guy would like to orally do to a woman’s naked derriere what many dogs would do to yours if you let them
• Many, many married men want a woman on the side, preferably someone younger, with firm breasts, a very high libido and with a predilection toward secrecy. Some will pay for the privilege.

Yes, there are “women” out there looking for men. Many are looking to “hit the slopes”, but their definition of skiing may surprise you. The vast majority though are run of the mill cyberspace pornographers trolling for email addresses. Others appear to be prostitutes who prefer to stay indoors instead of hiking their skirts near Logan Circle. They will not say it explicitly but it is clear enough with posting titles like “Sugar Daddy $$$$ wanted”. Those with a modicum more discretion are looking for “generous” men. For most though, sex, in the unlikely event it is realized, appears to be tangential. I guess they need all that sugar so they can keep hitting the slopes. To which all I can say is “Eew!” But I often say that when I pass an accident too.

Of the few ads from women that might actually be looking for a non-monetary and consensual sexual relationship, most are from BBWs. BBW apparently means “Big Beautiful Woman” which I suspect really means, “I make Edna in Hairspray look petite”.

Curiously, there seem to be lots of married women, almost all 30 or 31, who happen to live on Capitol Hill or Georgetown looking for a fling. Many claim to be wives of congressmen. Don’t worry, they manage to work out regularly at the gym, don’t smoke and promise complete discretion.

Of course, there are guys looking for other guys, sometimes married guys. I am not a homosexual but if I were, why would I be looking for married men? It seems like it would be more efficient to visit the local bathhouse.

Yes, there are plenty of libidinous couples out there looking to expand their humdrum sex lives. More power to them. Nevertheless, guys, they don’t want you. They want another woman. The wife is apparently very bi-curious, and the husband is just plain curious to watch his wife have sex with another woman. He no doubt also hopes he will get into her pants while his wife participates. The few couples out there looking for a man for a triad have a husband who usually asserts that he is not even a tiny bit bisexual. Oh please!

There are more women looking for women ads than I expected. Sixteen advertised so far today, and I would assume these are legitimate. Some cannot seem to admit they want sex. They claim they are looking for make out sessions only. Maybe they just want to stick one toe into the water to see if it is cold. Others can be as clinical as any guy.

It does not take much reading before these sorts of ads no longer seem the least bit remarkable. However, we discerning Craigslist readers look for not the routinely bizarre, but the desperately and unusually bizarre ad. What am I supposed to think of this posting that I saw today?

I want to empty a can of WD-40 into your…. - 25

I will travel to do this. I have my own funnel, so you don’t need one. I would appreciate going dutch on the WD-40, but not required. I would use a can of Right Gaurd because it smells better, but that cost twice as much.

Hell, if all goes well, I will drop a duece in your grill. No charge. All I ask is that you be discreet, rich, white, skinny, pretty, left handed, scorpio, and have a dog. Not cat people (a.k.a. freaks) please.

If this sounds like your cup of tea, get ready to start living. I mean it, too. It’ll be nothing but apple juice wishes and sardine dreams from here on out!

This one is clearly a joke, but with some it is hard to tell. At 48, I think of myself as something of a man of the world. However, ads like this, even when they are a joke, can still leave me bug eyed.

December 27th, 2005 at 10:01pm Posted by Mark | Sociology | 2 comments

The Elegance of XML

As those who follow my Technology archive know, Extensible Markup Language (XML) fascinates me. Finally, there is a standard and non-proprietary way for transmitting and receiving data of all kinds. These data, even in this trivial example may appear bloated. After all, every single piece of data must be wrapped in a tag that describes its meaning. This drawback though is also its biggest virtue. It makes XML both unambiguous and easily decipherable. The human eye can infer the meaning of the data in an XML document as readily as a computer. (The computer of course can process it much faster.)

Because XML is an open standard, there is no possibility of vendor lock in. Using programs called parsers offered by many vendors (often for free) XML files are easily sliced and diced by computers, thus allowing data to be transformed from one use into other use very easily.

If there is a true dark side to XML, it is that to share data widely, it really helps to get all the key players who understand a problem domain together to agree on how to describe their information in XML. For example, I work with hydrologists. However, my agency is not the only one out there that needs to describe the characteristics of water. So although we are engaging in our own efforts to describe surface and ground water in XML, we really need to have conversations with everyone is this arena. I hope that in time all parties will agree on a common set of schemas that will properly describe water’s characteristics. This will greatly enhance the ease on which data can be shared.

Since I am in the business of serving water information, I would very much like to liberate the data on our web site. It is currently largely embedded inside the HTML that makes up web pages. This is great if you are a human being looking at the information with your browser. However, it sucks if you need to grab the data and process it. I described some of my work in other entries. As we move away from looking at XML in the abstract to using it in real life, a few things strike me.

First is that XML can be collected in either complex or simple ways. Each has its virtues and both should be supported. The complex way of sending or receiving XML data is using a protocol called SOAP. (Ironically, SOAP stands for Simple Object Access Protocol. Prior to SOAP even more complex technologies like CORBA were needed.) SOAP has a few important virtues that can be summed up as follows: it supports transactions and error messages. You would probably want a confirmation that if you moved $1000 from savings to checking that the $1000 did not disappear. The SOAP protocol can effective say “Yep, moved exactly $1000.” In many cases when doing commerce over the internet, this kind of information is crucial.

However, it turns out that more often “quick and dirty” data dumps in XML are quite acceptable. For example, if you sent an XML request to Weather.com to return the current temperature for your town, you probably do not care too much if you do not get a timely response. If you do not get a response in, say, ten seconds you figure, “well, their server is down or slow”. If you are a programmer, you can write some simple logic to work around these situations. This is analogous to sending a casual letter to a friend or sending a certified letter. Not all data moving as XML needs to be wrapped numerous times to ensure safe shipment and delivery. You may not need to deal with all that overhead. As a response to these many simpler needs, web services for serving XML have actually devolved a bit. Protocols like REST make XML queries and responses quite simple.

Where I work we have been putting these XML transport protocols through their paces. Since we are a government agency, we look at other agencies as examples. “Lightweight” XML seems to be the direction most federal agencies are going. (NOAA has some excellent examples.) However, upon further investigation we discovered that some lightweight protocols could actually be heavyweight too.

Impossible you say? Not at all. What is happening is that the complexity of dealing and understanding XML does not have to rest with the data provider. The data consumer can also deal with it. The light bulb went off in my brain when we started experimenting with Keyhole Markup Language, or KML. KML is of course an instance of an XML schema definition. It is used by the popular mapping application Google Earth. It is trivial to annotate a point or an area on the earth in KML. Once you have done so, you can simply make the KML file available for download. Google Earth will read it and suddenly these new sites of interest will appear in Google Earth. If done right they will appear and disappear depending on the scale that you choose. What is handling this complexity? Google Earth is. Google Earth is really a very complex but excellent client computer program optimized for showing places on the earth that are described in simple KML.

It is not just Google Earth. Complex broad XML-consuming applications are popping up more frequently these days. To use another example many of you may be familiar with - a newsreader program. Yahoo! News, for example, makes its news content available in the now ubiquitous RSS (Really Simple Syndication) XML format. Your newsreader may be another web site like Bloglines, a program you download, or even (my favorite) an extension to your browser. For Firefox bigots like myself, the free Sage newsreader is a great way to grab these RSS news sources, aggregate them and present what to me feels a customized newspaper from sources that I care about. Again, though the RSS files provided by the host are very simple to create, it is up to a complex client computer program (the newsreader) to organize and present it. This pushes dealing with the complexity of the information to the client application.

Instances of XML schema definitions like KML and RSS demonstrate that lightweight protocols can do some very heavy lifting indeed. In my office, I have a programmer experimenting with making some of our water information available as a news feed. It could tell someone’s newsreader, for example, the current water height for their favorite local stream. The RSS news feed format, since it represents news, assumed that the key data in a RSS feed will be read by a human being. Therefore, an RSS news item might have a title like “Latest Station X Gauge Reading” and content something like “Station X on River Y reported a gauge height of Z at Time T”.

Could this technology that is used to facilitate human access to news sources also be used to support machine to machine data communications? The answer, surprisingly, is yes. Because as I dug into the details of the RSS 1.0 specification I realized that it also supports modules. While of course text like “Station X on River Y reported a gauge height of Z at Time T” could be parsed by a client program to pull out the essential information (Z and T), then store it and serve it locally, with a module this extra work is not necessary. Through the use of XML namespaces, the same RSS 1.0 news feed could incorporate a module that might include relevant tags like <hml:stage units=”feet”>4.5</stage>. Here is a way to assert a gauge height reading while piggybacking on a generic RSS news feed. By doing so, the news is still served by the newsreader, but the feed contains more discrete information for those who need it.

This may seem like nothing. I think this it is revolutionary stuff. A news feed is no longer just a news feed, but a news feed that could morph into multiple other uses. RSS is not just data on steroids. It is not just data married with its metadata. Using modules to piggyback discrete data to vanilla RSS, XML becomes even more powerful. It is like transforming a voice line into a DSL line. It becomes far greater than the sum of its simpler parts.

Observation: XML is both data and software. Yet it is not software in the conventional sense. XML is really a marriage of data (the values), information (the tags) and logic (the associated schema definition). I do not believe this has been done before in one taxonomy. In some sense then XML itself is also truly revolutionary. It is a completely new software paradigm, the implications of which we are just beginning to understand.

December 5th, 2005 at 09:54pm Posted by Mark | Technology | no comments

Intrigued by ActiveGrid

As frequent readers know, I was on the west coast this week. I was in Cupertino, California (in Silicon Valley) to attend a MySQL Customer Advisory Board meeting. MySQL is an open source and very fast database, used mostly on the web. Perhaps because we happened to have our representative pay a call a few weeks earlier I was invited to attend this meeting. As the only customer representing the federal government at the meeting, I was one of the dozen or so customers invited to throw in my two cents about the features that MySQL AB (the parent company) should put into the product.

Alas, I cannot give any hints on where the company will be taking the product in the years ahead. My participation was predicated on abiding by their nondisclosure agreement. I can say that if I were Larry Ellison (CEO of the mammoth Oracle Company) I would be worried. MySQL is a small but very agile company that has the lion’s share of the open source database market. It was cool to be in the same room with mega internet companies like Google and CNet and learn how they are creatively using the product.

A year ago last April I attended their MySQL User Conference in Orlando, Florida. It was there that I got the open source religion. Nothing since then has changed my mind. I think proprietary software is going diminish. I am sure Oracle will still be around in a decade. However, it might well be a shadow of its current self. Every software company these days should at least be pondering an open source strategy. At least one other database vendor has the religion. Ingres recently went open source.

MySQL won its market share the hard way: by creating a great and (in many cases) dirt free product. Corporate MySQL licenses do cost money but they cost a tiny fraction of what you will spend for products like Oracle. Most of their money is made in technical support contracts, consulting and training. However, their product is reliable enough that many corporations can do without these extras. I was interested to learn one tidbit at the meeting that demonstrates the difference between MySQL and Oracle. At MySQL, its founder Monty Widenius is one of three people allowed to put new code into their software baseline. Somehow, I cannot see Larry Ellison doing this. Heck, I am not even sure he could write a line of code. He is too busy flying places in his corporate jet or hanging out with his foxy wife. Having met Monty, he is certainly no billionaire. He is a geek with no pretensions of grandeur.

MySQL is one of the best of many, many open source products out there. Open source MovableType software used to run this blog. It is licensed a lot like MySQL: it requires modest costs for those who want more than a personal blog. However, much open source software is wholly free. Apache, the web server used to serve this web page is one example. The Linux operating system is also free, although most people prefer to purchase packaged distributions like RedHat. A number of programming languages for the web like Perl, PHP and Python are also robust and completely free.

Linux, Apache, MySQL and Perl/Python/PHP form a set of core open source products that offer amazing quality and features for little or no cost. They also work and play very well together. This synergistic combination of products is sometimes referred to under its acronym: LAMP. Mastering any of these products is not rocket science. This makes them affordable and accessible to the masses. Any reasonably smart person who has fiddled with a programming language can write LAMP applications by reading a couple books. (Writing professional LAMP applications is another matter.)

In addition to open source software, a new standard for data communication has emerged. Specifically, in 1998 Extensible Markup Language (XML) became a recommendation by the World Wide Web Consortium. XML may be a standard but in some ways, it feels like open source software. Vendors are busy writing software (much of it open source) that reads and writes XML. In addition to describing data in a standard way, XML specifications exist that allow data to be validated (XML Schema), rendered as output to various kinds of devices (XML Stylesheets), describe workflows (BPEL) and capture input (XForms). With XML as an industry standard, the cost of doing business electronically over the internet is going much lower.

It was probably only a matter of time before someone looked at the largely free LAMP stack, looked at the uses of XML and said, “Wouldn’t it be great if there were an open source LAMP software solution that could help you develop web applications quickly and also worked transparently with XML.” That day has arrived and ActiveGrid is the emerging product.

The system I manage is a perfect example of the hassles of using open source technologies the old way. Our system is a pure LAMP application but it is a pain to maintain. It was developed in the late 90s when this stack was just coming together. The result is a wonderful system that is amazingly flexible but a pain to modify. Perl, our primary programming language, was not designed to be object oriented. Object oriented languages are important for modern systems that need to be designed to morph over time. As the system manager, I am looking to the future and imagining how the open source system of the future should be engineered. ActiveGrid, if it proves viable, looks to do for applications what MySQL has done for databases.

All the software ActiveGrid generates is open source. You may have to write some of it by hand. If you do then you pick Perl, PHP or Python, whichever rocks your boat. (PHP support is coming.) MySQL can be your database, but you can also use other open source or commercial databases including SQL Server and Oracle. ActiveGrid assumes that your web application will be built on top of Linux and Apache. All these open source LAMP technologies run on cheap commodity hardware. Therefore, it is not difficult to stand up enterprise class web, application and database servers, each on separate machines, for $20,000 or less out of pocket. The economics of this LAMP model are compelling.

ActiveGrid allows you to build LAMP applications quicker. It provides an abstraction interface for many of the things you would have otherwise code. For example, you can design screens using its drag and drop tool. This is a lot faster than creating the code in an editor! You can design your workflow logic graphically using a tool that renders open source BPEL. What is particularly cool is that bundled in the ActiveGrid toolset are a number of XML engines. Therefore, your users might see web pages rendered as HTML, yet under the ActiveGrid hood, it has used XML stylesheets to render data described in XML as HTML. Slap other XML stylesheet templates on these data and they can become PDF documents, rendered for a cell phone or sent in a format designed to be heard (VoiceXML). On the input side, the user might see a HTML form, but it is translated into XML using XForms technology. Supposedly, the tool is sophisticated enough to render AJAX compliant code in the browser. This potentially gives it a very robust web interface such as you can see in Google Maps. In addition, much of the business logic is handled by an engine built into ActiveGrid that reads BPEL. In the past, you had to writing a lot of Perl, PHP or Python code to implement business logic.

As I have mentioned before, systems basically take input, apply business rules to it, and render it as output. ActiveGrid does this with XForms, BPEL and XSL. The result is one tool that leverages the low costs of the LAMP stack, commodity hardware and open source XML toolkits. ActiveGrid has the potential to create impressive web based systems that are quick to create and deploy yet are entirely open source. Technologies like Java 2 Enterprise Edition (J2EE) will continue to have their place. However, with ActiveGrid these open source technologies can be orchestrated to render complex applications that are likely to be just as reliable and efficient for a fraction of their cost.

How will ActiveGrid make money? Already an open source version is available, though it is still being tweaked. ActiveGrid will introduce in December a commercial version with fancier features. It is clear that the standard product is ample for most needs. However, if you want features like integration with Enterprise Java Beans (EJB) or Lightweight Directory Access Protocol (LDAP) then you need to pay for a commercial license. The proposed price ($3K per server per year) does not sound very burdensome. They also will offer paid support for both their standard and commercial products.

As I am a manager, I do not have the time to do much programming. However, I am hoping that in my spare time I have more time to play with this product. I want to put it through its paces. Moreover, I want to find out whether it can also easily create genuine W3C complaint web services. If so, this might well be a great product to use to create the next version of our system.

If you have experimented or deployed systems with ActiveGrid, please leave a comment and let me know your experiences with the product.

October 7th, 2005 at 10:14pm Posted by Mark | Technology | no comments

The telephone transformed

My web cam arrived today. I ordered one for myself, along with a headset. Both plug into the serial ports of my computer. Since I am a manager, I also ordered sets for my whole team. The time has come to say goodbye to the telephone.

My team, like many nowadays, is geographically disbursed. I have three employees working with me in Reston, Virginia, with others in Portland, Oregon, Helena and Anchorage. As you might expect we already do a lot of our work using the ubiquitous telephone system. At a minimum, we have two conference calls a week, each lasting at least two hours.

Until now, the telephone system has been an essential collaboration tool. Its simplicity can be a virtue, particularly when the network is experiencing hiccups. However, the reality is that we cannot do our work solely by telephone. Like all knowledge workers, we cannot be efficient without standard Internet technologies like email and instant messaging. IM was theoretically prohibited for years in our agency, but we found we could not do work without it. The AOL Instant Messaging (AIM) program somehow ended up on all our desktop computers.

Until about six months ago, our conference calls worked like this: everyone looked at the same web page that had our agenda on it. The scribe would take notes, and save them periodically. We would refresh the web page with our browser to verify the notes and take down our action items.

About a year back, my team leader and I started playing with Microsoft NetMeeting. This Windows program lets two or more people share their computer screens live and in real time over the Internet. We found that it worked great and began using it with our conference calls. Since that time, my agency has invested in hosted virtual meeting solutions that are a bit easier to use. We have joined the numerous list of Webex customers who meet online. Note pages are no longer static. We watch the scribe as he or she types up the notes in near real time. Sometimes the scribe will switch to another application, like Microsoft Project, and we can all see the project schedule and give input to it. To work efficiently, these “live” virtual meetings depend on our fat pipes to the Internet.

The fidelity of telephones, to put it mildly, sucks. Companies like Skype are taking advantage of the Plain Old Telephone System’s (POTS) drawbacks. They figure millions of people are already sitting in front of their computers all day and many also have headsets and microphones. Therefore, Skype lets them “phone” each other over the Internet. As long as their bandwidth is high and the network latency is low, the voice over Internet experience is dramatically better than using a telephone. It is like hearing someone sitting next to you, instead of hearing a voice through a walkie-talkie. Skype offers this service for free to those who are comfortable with computer to computer voice communication. They will also be glad to connect you to someone on a landline if necessary, but that is when their meter starts running.

Skype’s business model seems to have attracted attention. Ebay is planning to purchase Skype. It recognizes that there is an emerging market for voice over the Internet. It is hardly alone. There are all sorts of companies in variants of this business. Vonage is one of many. It offers what appears to be a regular telephone service, but all conversation is done digitally over the internet.

Services like Skype suggest that we may be paying too much for telephone service. Indeed, Skype seems to be saying voice over Internet to anywhere in the world should come free with Internet access. Increasingly our computer networks are becoming faster, cheaper and more reliable. At some point in the not too distant future, the landline will become obsolete. In fact, many people have already replaced their landlines with a cell phone.

Yet even cell phone networks may be obsolete in a decade. If phone calls can be made over the wired Internet then why not over a wireless internet too? While it is possible to see web pages using a cell phone network, it is not necessarily the best way to do it. Using a cell phone network to access the web is sort of like using 9600-baud modems to access the Internet. Just as voice lines were not designed to carry data, cell phone networks are not optimized to deliver Internet content. However, a new wireless Internet infrastructure is emerging. Verizon Wireless Internet, for example, claims to be accessible to more than a third of all Americans. Of course, you need to be near a metropolitan area. With Verizon Wireless Internet, you do not need to find a local hotspot to get online. As long as you are in a broad service area, you can get 400-600 kilobits per second wireless internet access. This is comparable to DSL speeds.

In the office, where 100 megabits per second is standard, 400-600 kbps is pokey. With this kind of bandwidth, voice and even video over the network is viable. That is why I ordered computer headsets and web cams for my team. Why not? I am sure my agency pays a heap of money for our data network infrastructure, but probably does not pay for the amount of bandwidth used. VOIP (Voice over Internet Protocol) technologies deliver voice over the Internet, but in a way that guarantees no noticeable delays. (The Internet is a packet switched network, and was not designed for synchronous communications.) Since my agency does not yet have VOIP services, voice over the Internet may at times be a bit choppy. Until VOIP arrives, we still have POTS that we can still use if needed. Nevertheless, I believe that we will prefer voice over the Internet instead, because latency should be minimal and we will appreciate the better voice quality it provides.

The web cams we purchased will also come in handy. I can use them to find out if someone is at their desk or has someone in the office. I can even see their body language. This is impossible with voice only networks.

Both web cams and computer headsets are cheap commodities. Therefore, purchasing them seemed a no-brainer for me. The web cams and headsets have not all come in yet. Once installed I anticipate that we will improve our productivity and lower our costs too. As for POTS, it is already disappearing. In fact, most long distance POTS telephone calls are already turned into a digital format for transmission. At the local telephone exchanges, the digital calls are transferred back to the familiar analog signals that Alexander Graham Bell first used.

The Baby Bells, sensing the end to their reign, are busy putting fiber optic networks in place for next generation data services. Telephone and Internet services are converging, but telephone services will simply become another feature that will be transparently handled over the Internet. The flexible and efficient digital nature of the Internet is bound to win this battle. The telephone as we know it will soon fade into obsolescence, as puzzling to the next generation as typewriters are to Generation Y.

Suits me.

September 23rd, 2005 at 10:28pm Posted by Mark | Technology | no comments

We were cyberspace pioneers once, and young

In 1984, I purchased a Commodore 64 computer and a 300-baud modem. I cannot recall exactly how much money it cost me. I think it was around $300. For the money, not only did I have a “personal” computer but also I had a way to reach out into a new electronic frontier. With my girlfriend (and subsequent wife) looking over my shoulders, I connected my C-64 computer (with its whopping 64K of memory) to my phone line. I then dialed long distance to another computer in Arizona. I found the phone number to dial from a newspaper article. This was my first adventure into cyberspace. I was 27 years old.

The C-64 was nothing fancy. Nevertheless, for those of us interested in dabbling in computer science, it had great virtues as a learning tool. Between the Apple 2e at work and the C-64 at home, I learned far more about computers than I ever expected. When programming in BASIC was no longer enough I experimented with assembly language. I even wrote some self-modifying code. These plain machines were almost as addictive as crack. Eventually I succumbed. Information technology became my second career. Now it now keeps me very comfortably out of poverty.

The C-64 eventually proved limiting. However, I could not afford an IBM compatible PC, which were running at least $2000 in those days. My employer at the time gave me a loaner computer so I could work from home if needed. It ran an 8086 processor and MS-DOS 3.1. The modem burned up the phone lines at an incredible 1200 bits per second. My monitor resolved green letters against a black background. I could use it to dial into work. However, I could also use it to dial into IBM PC compatible electronic bulletin board systems. There I could download programs and games. After a year or so that got somewhat old. Instead, I started to spend more time in the bulletin boards actually chatting with fellow BBSers.

BBSers. BBS = Bulletin Board System. We denizens that spent way too many free hours connected to electronic BBSes were BBSers. In those pre-Internet days, those of us who could not afford AOL or Compuserve dialed locally. “Sysops” put their PCs online and used neat software like PCBoard to capture conversations and share files. We chatted with people who at first were virtual strangers. However, over time they emerged as real people. Moreover, they lived locally. Most used aliases. After a while, you learned the cool BBSes to hang out. Armed with Mike Focke’s monthly updated lists of local BBSes we dialed around a lot. It was like bar hopping, except you never had to leave home.

I was not beyond my high hormone years so I spent a year or so on adult BBSes. I was married by then but it was okay with my wife. She even tried BBSing out for a couple months before she lost interest. Locally there was a naughty BBS called The Doctor’s Office featuring “Doc” and “Nursie”, a married couple who were also swingers. If you wanted then instead of leaving dirty messages to strange women, you could go into chat mode and have this thing called cybersex.

This was very novel in the late 1980s. Hopefully the person you were having cybersex with was of the correct gender. (Women were hard to find online in those days, but somehow Doc and Nursie found them.)

Since Doc and Nursie were local, I went to a couple of their parties. No, I never got naked. I never came close. Some of that did happen from time to time, usually after midnight and in somebody’s hot tub. I did not hang around that long. As for the parties, I found that they were somewhat boring. Many people on the board turned out in real life to be smokers, or obese or dealing with many mental issues. So within a year I lost interest in the Doc and Nursie’s board. However, there were other boards with adult areas (like Zonzr) that attracted a better class of clientele. Under an alias, I had a lot of fun seducing women over the course of conversations that stretched for weeks. No, I never seduced someone this way that led to a real sexual encounter. Actually, I found the idea of online seduction in cyberspace more attractive than following through.

After a while I found that I was much more interested in spending my time on BBSes talking about politics than I was doing the electronic equivalent of heavy breathing. The reality was that sex chat got boring quickly. In addition, I was blessed with a wife who was a pretty sexy creature, and who was broadminded enough to let me indulge myself online. Since it was never illicit, it was a bit like drinking flat champagne. However, there were plenty of BBSes in the Washington DC area that offered political discussions. I was referred by a friend I met on Zonzr to Dawn Gibson’s very private little BBS called The Back of the North Wind. There I spent nearly nine years keeping the place lively. Also on The Back of the North Wind, I met people who would turn out to be my friends.

We would get together periodically, sometimes at someone’s house and sometimes at a public park. Riverbend Park north of Great Falls on the Potomac was a favorite place for get togethers. The BotNW people were fun, intelligent and very eclectic. I liked the exclusive and private feeling of the place. Dawn did not allow aliases on her board. She did however get very tired of the political discussion. So she created her own areas of the BBS featuring creative works including short stories and poetry. Dawn was always blessing us with her latest poetry.

Then the World Wide Web happened. By 1995, the end of the BBS era was easy to discern. Dawn kept the BBS going until 1998 anyhow. Her board was probably one of the last to disappear. However, by then it seemed something of an anachronism. Why dial up weird phone numbers and wrestle with terminal emulators when you could simply open up Netscape, jump to Yahoo and surf to whatever caught your fancy at the moment?

A new era began and an old era closed. Nevertheless, there was something unique about BBSing that the Internet does not really provide. BBSing allowed me to find people with similar interests who were also local to me. It was cool to meet some of these people in person from time to time. People who would never run across each other in real life could become friends. In addition to Dawn, my real life BBS friends included Jim, Frank P., Frank S., Debby, Angela and many, many others.

As the BBS age drew to a close it seemed likely that most of these friends would disappear into the vastness of the Internet. Fortunately, I had some of their email addresses. Some took the opportunity to close that chapter of their lives. Others nibbled around the edges. Still others, like me, missed those days and sought to recapture them. After The Back of the North Wind closed down, I tried substitutes like local Yahoo message boards. However, it was not the same. The Internet was full of foul-mouthed bozos with 8th grade educations who were as deep as a baptismal fount. Finding good conversation was tough. Finding good and local conversation on line became nigh impossible.

I was nostalgic. Eventually I decided to create my own place in cyberspace to try to recapture those glorious BBS days. It is called The Potomac Tavern. It has been on the web about five years now, two of which were on ezBoard. Eventually I discovered phpBB and decided to host the darn thing myself. If you have an interest in general conversation with an emphasis on politics and have the perseverance to visit regularly, perhaps you should check us out.

Trying to keep the local feel of a BBS in the Internet age has been difficult. The Internet makes location irrelevant. Nevertheless, since many of the members are people I knew from my days BBSing, many of them are happy to hang out on The Potomac Tavern. There are about a dozen of us regulars there saying our peace. It is small, but a fun place on the Internet. We have liberals and conservatives. We talk nice to each other. We do not tolerate rude people. Many of our conversations have real depth. Although I once had pretensions of some bigger and grander place in cyberspace, I am now quite content with our regular crowd of intelligent and artistic people just chatting about whatever is on our minds.

Why all this ancient history? I mention this because sometimes you can recreate the past. While The Potomac Tavern has its own unique feel, somewhat similar to what Dawn Gibson created on The Back of the North Wind, and many of the same members, it is still a different kind of place. Frank P., Angela, Jim and Debby — friends I made in those BBS days — now hang out in my forum. Others, mostly friends of Frank, have also joined the forum and contribute everything from poems to daily facts to new discussions.

We recreated the past on Sunday. I got a number of my old BBS friends to show up at my house. While a few had to cancel, and a number who did attend do not actively patronize my forum, it was still a fun time. It had been at least seven years since we did something like this. There were eight of us altogether. They provided dishes, I provided a grill and meets. We drank beer. We laughed. We talked about Hurricane Katrina and day laborers. We ate cheesecake. All of this happened on my deck, screened from the bugs, on a perfectly glorious Labor Day weekend.

One thing has changed. We are older. In some cases, we are also wider. Twenty years ago, I was young. Now I am pushing 50. In addition, I was one of the younger people at my own get together. Most of the attendees were now in their 50s and 60s. However, it did not matter. Time rolled back. We were still the fun, interesting people who helped create cyberspace twenty years ago when it was uncool and geekish. Places like The Well and Yahoo Personals owe at least some of their success to us cyberspace pioneers.

Yes, we were cyberspace pioneers once, and young. Moreover, despite the odds at least some of us remain good friends after all these years. We still enjoy each other’s company, in person when we can arrange it, but mostly online. After twenty years, I realize that I have lost the artificial distinction between friends who are online friends and friends who I know in real life. They are all my friends now. Even those I only meet virtually can be as dear to me as those I know in real life.

We were among the first astronauts of the Internet age. If we have inscribed on our tombstones “cyberspace pioneer”, it would certainly be fitting.

September 8th, 2005 at 08:52pm Posted by Mark | Technology | 3 comments

Why Occam’s Razor is like the Oxpecker Bird

So I’ve been monitoring my web logs, as I often do. I am seeing referrals from washingtonpost.com. WTF? Washingtonpost.com is promoting my backwater little blog for no money? I don’t think so! The referring URL is actually to a page that I linked to in one of my blog entries. My blog entry (written first) pointed to an article on its web site. It seems that the Post knows that I have linked to it. Moreover, on the same page as the article itself it is highlighting my blog entry! (Here is an example. Look for the “Who’s Blogging” box lower in the page. Bear in mind my blog entry may scroll off their page after a while.)

How is this magic happening? It is thanks to Technorati, a blog indexer. When I publish a blog entry, I automatically ping the Technorati site. Apparently, Technorati reads the entry and then indexes it. Moreover, either it informs washingtonpost.com, or washingtonpost.com asks Technorati during the day what blogs are linking to its articles. Washingtonpost.com then serves my blog entry dynamically as a related blog link on the article page. The thousands of people hitting that page then may choose to read my related blog entry.

Many web sites these days are using the power of Technorati. (Technorati is sort of like Google for blogs. It calls its index “The World Live Web”.) However, washingtonpost.com has pushed the limits by serving recent related blog entries dynamically right on the article page itself. This is, of course, a variant of the Trackback technology pioneered by MovableType, the people who let me steal their blogging software. (Since I only host one blog, I can use it for free.) You can see Technorati in use on other sites like Salon.com.

To which I say, way cool! My blog could always use more page views. In addition, if I go through all the trouble of finding a related link on the washingtonpost.com site, why shouldn’t they link to me too? It should be a win-win situation. I draw some small amount of traffic their way, and they serve ads when my readers click on links directing them to their site.

But … but I know what is going to happen. It is too good to be true. This will soon become the spammer’s next frontier. Soon bogus blogs will be springing up faster than dandelions in the spring, all referring to prominent and recent Washington Post articles on Washingtonpost.com, or on other newspaper sites highlighting this technology. Presumably, the Post or Technorati have thought about this already, and are only serving trusted blogs.

Good luck to them, but it cannot last. I have noted before that spammers are evil, but ingenious. There is already blog entry generator software out there designed to fool search indexes and news aggregators. They create bogus blog entries with sufficiently interesting words that (they hope) will fool the search engines. They hope that Google and Technorati will not be able to tell real blog entries from the fool’s gold. This evil software in theory lures suckers to their sites and gives them, along with the bogus entry, a heaping dose of advertising. Yes, you too can make a million dollars off the internet, or so these companies claim.

Search engines and news aggregators will doubtless attempt to create ever more sophisticated technologies to get rid of these tomfooleries. Nevertheless, I can see that this will create an ever more vicious cycle. When that happens, then sites like washingtonpost.com will realize they are being suckered, and will stop featuring blogs like mine with related content.

In fact, this particular misuse should appear faster than an alcoholic can retrieve his hooch from behind the couch. I have not examined these related blog links, but I bet some spam links are already out there. After all, washingtonpost.com gets millions of page views a day from all over the world. Placing ads on their site costs serious money, typically purchased for thousands of dollars at a time. Why pay for advertising when you can get it free by creating bogus blog entries?

Therefore, I am betting that washingtonpost.com will show related blog entries for a month, maybe two, before the spammers spoil yet another great idea. Then I will be back to finding readers the old fashion way: through word of mouth or through fortuitous search rankings.

However, while this lasts clever bloggers looking to increase their page views and visits might think through an effective strategy. Thus far, most of my entries featured on the Post’s web site are for articles days or weeks old. Consequently, they tend to show up on the article’s web page itself, rather than indirectly through the “Full List of Blogs” link on the article page. If I were to blog about their top story of the day, I suspect my entry would not get prominently featured for long. It would soon scroll off the page because someone else would create a blog entry that is timelier or perhaps more interesting than mine.

I hope I am proven wrong. This is neat technology. Just as the oxpecker bird removes insects from the skin of the hippopotamus, it is both right and good that bloggers and content providers can also find symbiosis. However, you would be foolish to bet against the spammers.

September 6th, 2005 at 08:32pm Posted by Mark | Technology | no comments

Robots and Spiders and User Agents - Oh my!

Six weeks or so ago I complained about SiteMeter. I noticed big discrepancies between the number of visits and page views that it was reporting versus what my Apache server web log was showing. These discrepancies remain true. However, as I dig into the details of my web server logs I am beginning to realize that it is impossible to know how many people are looking at my site. Moreover, the same is true if you own a web site.

Apache does an excellent job of recording every hit to my site, as you would expect. The problem is separating the wheat from the chaff. Ironically, my investigation suggests that getting an accurate measure of a blog’s audience is due to news syndication technology that I have praised.

For example, this site is hit twice an hour by bloglines.com. Bloglines.com is a news aggregator. It sniffs through blogs and apparently caches content (i.e. stores a copy of my pages) on its servers. If a bloglines.com reader has subscribed to my blog, bloglines.com will present their local copy of my latest blog entries. I have no way of knowing how often a bloglines.com subscriber is reading their cached copy. However, I have some idea of how many bloglines.com subscribers are looking at my blog. That is because bloglines.com is nice enough to tell me the total number of subscribers in the user agent field. For my index.rdf newsfeed, for example, it reports 11 subscribers. I have one subscriber to my atom.xml newsfeed. Nevertheless, how often each bloglines.com subscriber looks at my newsfeed is unknown. Perhaps bloglines.com will someday share its statistics with us content providers. I did not see any information on their website suggesting they will be doing anything like this.

Bloglines.com is just one example of a news aggregator service. There are dozens of others. It is difficult to determine which are genuine aggregators and which are vanilla robots trolling my site. Arguably, they are the same thing. A robot is simply a program sniffing through my site for content and perhaps storing some of it in a search engine. A news aggregator agent does the same thing, but presents the content to people specifically interested in viewing my content remotely.

When a robot hits my site, clearly no human is reading the content. At some future time, a human may select a link through the robot’s web site that may take them to my site. Alternatively, like Google, they may be reading a local cache of my web page instead of hitting my site for the real thing. There is no way to know for sure but most likely, I am missing statistics for many page views.

There is also a lot of “noise” in my web server logs. I did a number of Google searches on some of these user agents. I discovered that many of these agents have malicious intent. They are looking for various vulnerabilities on my site, presumably to inject some viruses or Trojan horses. My own hits to my site are in the log, along with the program I wrote to examine the log. From my perspective those hits are noise to be filtered out. There are also many requests for pages that do not exist. All these need to be filtered out too.

In addition, new robot programs are unleashed all the time. Yahoo, for example, has two going. The normal one is Yahoo Slurp, but there is also Yahoo commerce robot called Yahoo Seeker. Yahoo also offers a news aggregator service. Unlike bloglines.com, its user agent string does not identify the number of subscribers. So once again, a webmaster like me can only shrug his shoulders.

There are many robots out there apparently. At some point, I am going to have to fine-tune a robots.txt file. This file is supposed to tell search engines the content that may be indexed on your site. Reputable robots obey the robots.txt file. Shady robots — and there are apparently plenty — do not bother and will index your content anyhow. To keep out the shady robots you have to fine-tune an .htaccess file to block them. This is generally a manual process, and thus avoided by many webmasters.

Overall, I think it is fair to draw a couple conclusions. “Metadata” is growing. Metadata is information about a website, and robot programs on remote computers stream across the Internet to your web site to collect it. Any kid in his basement can write his own robot program and start trolling your site for content (and suck up your bandwidth for no compensation). Robots are equal opportunity programs. As metadata grows, it becomes more important for a webmaster to be able to manage metadata collection. At some point, I expect there will need to be mechanisms more sophisticated than Apache .htaccess files. Perhaps web hosts will offer sophisticated application level filtering, allowing in a known list of reputable robots. Alternatively, perhaps they will let in only those robots that share their metadata. For a webmaster like me, this seems eminently fair. How many subscribers are trolling my site via your site? Who are they and how often do they come? If you are going to sift through my content the least you can do is accurately identify yourself and tell me how many people are reading my site and how often. I can see the Internet Engineering Task Force or the World-Wide-Web Consortium creating some standards for exchanging metadata statistics in the near future.

Second conclusion: the browser if not dying, is not exactly well. Looking through my web server logs I see all sorts of applications are reading my site. Not all are apparently robots. For example, there is the Microsoft URL Control. I Googled it and discovered that virtually any Microsoft program that uses the Internet uses this control. Therefore, someone clicking on a link embedded in a Word document is probably using this control. There are also Java programs talking to my web site. I see Java/1.5.0 and Java 1.3.1 as user agents. Whether they are more robot programs or application programs serving my content is impossible for me to figure out.

The bottom line is that it is becoming impossible to know how many human beings are reading your site. This is not good. For larger sites, this makes it difficult to provide meaningful metrics to advertisers who might want to pay to place ads on your site. It also leaves the webmaster scratching his head: is my site popular or not?

Perhaps it is close enough to take a SiteMeter reading and multiply it by some number to estimate the total readership. On the other hand, perhaps a webmaster should take his page views in his Apache web log and multiply it by some fraction. Whatever. It is a gross estimate at best. Perhaps the better metrics are simply a site’s Google or Bloglines.com ranking. Investigating page views and calculating visits is increasingly irrelevant.

July 30th, 2005 at 11:29am Posted by Mark | Technology | no comments

Discovering the Living Earth with Google Earth

I am rarely amazed. But Google Earth is truly amazing. It astounds me.

Google Earth is a program that you install on your PC. It gives you an incredibly easy way to zoom in on any part of the planet. It is amazing how quickly and easily you can go from outer space to zooming in on someone’s backyard. It is incredible how quickly you can focus on an address of interest. It is simplicity in itself to zoom out or pull back (use that middle mouse roller). Once you have marked a spot on the earth, it is transparently easy to find it again; particularly when it shows up as a focus spot on the globe if it fits on the map.

Perhaps the product is named wrong. It should be called Google Magellan. Ferdinand de Magellan was the captain of the first ship to sail around the world. That was an amazing act. Now you too can sail around the world in a virtual airplane or spacecraft and never leave your chair. You can zip from point A to point B with incredible ease. Moreover, as you do so you see the earth move around you in three dimensions. It is the first product that truly lets you appreciate the breadth and scope of our planet.

Of course, we have seen satellite and aerial photos on the web before. Google Map satellite photos are pretty darn easy to use: just drag in the direction you want, and zoom in on whatever is in the center of the map. Microsoft Terraserver has amazing detail for much of the United States. However, Google Earth goes way beyond even the recently introduced Google Satellite Maps. It seamlessly makes finding any defined point on the globe almost effortless. In addition, it provides convenient layers that show road names, local terrain, lodging, dining, buildings (in three dimensions in many cities) and borders.

It is an incredibly synergy of form, function and information. It is tethered to the whole massive Google infrastructure. To make it work you need high speed Internet. You also need a newer version of Microsoft Windows (Mac version under development) and a supported 3D graphics card. Still many of us have this already so there is nothing to buy. You also have to download and install the Google Earth program. Today this may be impossible. Even with its massive bandwidth and infrastructure, Google is having trouble keeping up with the demand. So as of today, you cannot download it. I got lucky. After reading an article about it in Sunday’s Washington Post, I downloaded it. Since then it has become almost an addiction.

How do I love Google Earth? I love the drag and drop interface. Once I have found my addresses of interest, I just drag them into the My Places area of the interface and they are remembered forever (including the height, perspective and layers I used when I last visited). I love the tilt feature. It gives me a forced perspective I never had before. For example, I live a few miles from Washington Dulles International Airport. Now I can soar above it and traverse the runways just like an airplane captain. It gives you detail you cannot get from Flight Simulator. Yes, I am very much like a bird in flight when I navigate with Google Earth. Unlike a bird, I do not have any altitude limitations. I can travel hundreds of miles a second. In addition, I love clicking on the roads layer and seeing the road names pop into place. I like the way the road names shrink or expand to fit the available space, or transparently go away if the type size would overwhelm or underwhelm the road. I like being able to rotate perspective on a dime. I like knowing precisely the latitude, longitude and height above the ground I am at during any given moment. I love the way when I go from point A to point B that it pulls back perspective and zooms me effortlessly and smoothly across the landscape, then zooms in to your new address. This is not just cool. This is not just way cool. This is ultra cool.

Google is partnered with Keyhole, which provides a lot of the digital imagery magic. Unfortunately, you cannot necessarily zoom into your backyard. The highly detailed imagery is not (yet) available for the whole country. But if you live in or around a major metropolitan area then the chances are that you will see your house, your driveway, your cars in the driveway, your trees, your bushes, even your fences. You can get a clue as to how detailed the imagery is in a particular area from the density of the grid. If it is more gray than green, for example, it is probably contains highly dense imagery. Certain states like Indiana and Massachusetts seem to have complete high-density images of their states.

If you wonder where the imagery comes from, it will tell you the source of the information near the bottom of every image. I was looking at Saigon last night and was a bit surprised to find that the imagery was provided courtesy of the National Geospatial-Intelligence Agency, an agency of our Department of Defense. I used to work there in the quaint days when it was called the Defense Mapping Agency. My newest employer, the U.S. Geological Survey provides a lot of the coarser imagery through its Landsat and EROS satellite images. If I have some impatience with the program, it is that I want high imagery photos for the entire planet. But it’s not there. Rest assured that if Google can get them then they will make them available.

Of course the first thing I did was examine from the air all those places where I used to live. But I’ve also had lots of other fun. I’ve looked at airports I’ve flown in and out of. It’s fun to locate them myself by zooming in on a city the navigating to it with my eye. I’ve found things with my eye that I did not think I could find. I found the hotel where I was at in both Montreal and Denver. I found a shortcut that gets me around traffic coming home from work.

While an excellent tool, it is not always perfect. It misplaced one of my residences by about a block. It put a gas station I frequent half a mile up the road. But at least 95% of the time it locates addresses or places correctly. Considering the volume of them out there that is very impressive.

When I was a youth and imagining the future, I figured that we’d have colonies on Mars by 2005. I could only dimly imagine personal computers, cell phones or even the Internet. But none of these things were as far out as Google Earth. I never expected a tool like this in my lifetime. This is Star Trek stuff. That it is available here and now and for free just blows my 48 year old mind.

If you are geography nut as I am then you will find that Google Earth is an addicting tool. Even if you are not you may find that with the easy use of Google Earth you will turn into one too. Moreover, perhaps like me you will get an appreciation for the breadth and depth of our planet. Yes, it is finite, but it fits together in both micro and macro like trillions of tiny puzzle pieces. With Google Earth, you may feel the earth come alive as a living entity for the first time. And for me that’s why it is truly amazing.

July 6th, 2005 at 09:50pm Posted by Mark | Technology | no comments

Webdrive: Integrated FTP for Windows

As a webmaster naturally much of my work is implemented on web servers. Clearly web servers are not desktop computers. And since most of my web sites are virtually hosted I have no direct access to the web servers themselves. Heck, in most cases I don’t even know where the web servers are located. Presumably they are somewhere in the United States but for all I know they could be sitting in server rooms in India. But somehow I have to place the content I create on my web servers.

So for years I’ve been doing it the same way everyone else has been doing it who have to do these things remotely. I’ve been using File Transfer Protocol. When I do it at work and the servers are actually located somewhere on my campus I have better alternatives for connecting to these machines. If they are Windows servers then it’s no problem: I just map them like another drive letter using My Computer or Windows Explorer. If they are Unix or Linux machines there is usually no problem either, providing SAMBA is installed on the server, as is typically the case. With SAMBA these machines transparently appear as Windows machines even though they are not: Windows is faked out.

But when working from home the choice has been pretty simple: File Transfer Protocol or some secure version of it. File Transfer Protocol is all about, well, transfer. Basically you copy the file you want to edit down from the server to your PC. Then you edit it. Then you copy it back up. It’s not difficult, just tedious and manual. Normally you have to authenticate yourself to the web server to get access, which adds time. Often you must manually traverse directories to get to the file you want. In short, while it is not difficult, it is time consuming if you are editing more than a couple files, or you need to upload files frequently. Using FTP connections can time out fairly quickly. In this case you often have to go through the hassle of connecting manually again.

It would make life a lot easier if these remote servers could be mapped transparently as if they were another Windows drive letter. Then I could simply open up these files in my favorite text editor and it would seem like the files existed on my PC. Fortunately, I have found at least one product that makes accessing remote servers via FTP or Secure FTP appear as just another drive letter.

The product is called Webdrive and it is a product of South River Technologies. Alas it is not free but it is also not prohibitively expensive. It costs $49.95. I have a license for it for those times when I need to work from home. I find myself more and more also using it for my remote webmastering duties, not all of which are directly related to my work. What a convenience it is! Now that I know about the product and use it regularly I know that I would pay for it out of my own pocket if my employer hadn’t picked up the tab.

I suspect there are other similar products out there, but I doubt very many of them also work with Secure FTP. That can be important since more and more web hosts are requiring the use of Secure FTP. As you might suspect FTP itself is not secure. IDs and passwords are transferred as plain text across the Internet.

So Webdrive is still using FTP, it just makes it transparent to you. Consequently there is no fast saving of files on remote machines. Files still have to traverse the Internet in order for them to be saved. It can often take 5-15 seconds for the file to transfer. Sometimes with a really bad connection or a really big file it can take even longer. But ooh the convenience to access it by simply doing a File/Open in my Windows application of the moment.

Webdrive can be configured to automatically remap the drive when you log on to your PC. It also does a pretty good job of keeping your FTP connection from timing out. If it does time out it will do its best to transparently reconnect you. My experience is that it does not always do it perfectly. It is only as good as the network on which it is running. So if your ISP is having access problems, or the server itself is slow you will notice performance degradation.

Webdrive is an example of one of these obvious solutions to ordinary problems that for some reason got largely missed. It’s such a common problem that you wonder why Microsoft didn’t build it into the operating system. (Of course the real issue is why Windows uses the drive letter metaphor in the first place. This is really a solution to address a fundamental Windows architecture problem.) But at least there is a solution that works. It only took an hour or two of my time before Webdrive paid for itself. It’s already paid for itself many dozens of times.

So if you do a lot of editing of files on remote servers over the Internet accessed by FTP then this is a product to definitely consider. Of course you can download a 20-day trial version that will give you plenty of time to decide whether it is worth shelling out $50 for a purchase. For myself, this is a no-brainer.

April 17th, 2005 at 09:32pm Posted by Mark | Technology | no comments