McWalter.org :: Blog index page

Apparently Anonymous, that vaguely extant hacktivist group that lazy journalists like to write about when they're Googling for news rather than going out and doing their actual job, has threatened to declare "cyberwar" on the fancifully named Democratic People's Republic of Korea [link].

Cyberwar on North Korea? That seems as likely to be effective as declaring cyberwar on House Lannister.

It's at this time of year that millions of college-age Americans return to the family home and to their childhood place in the family structure. Some can decide whether to sit at the kids' table or the adults'. If they choose the adult table, they'll hear an hour-long lecture about how Barack Obama is an evil alien who is out to destroy America; if they sit at the kid table, they'll hear an hour long lecture about how Megatron is an evil alien who is out to destroy America.

Does this mean that Barack Obama is really Megatron?

Apple's page about the replacement program calls it a "safety risk" without providing much detail; Wikipedia's article mentions a few overheating events (but really not a lot). Still, Apple are clearly worried about their liabilities. So they took back the old one (a black 2GB model) and they've sent a new 8GB silver one.

It's slightly strange to get an Apple product like this. There's none of the usual Apple "unboxing" experience, because there's no box at all. The Nano came in a generic shipping box (one downright Brobdingnagian when compared with the tiny size of the player). No manual, no cable, no software, just the tiny Nano with a serial number sticker.

I'm surprisingly sentimental about technology. I still have every mobile phone I've ever owned (though I've given one to a relative) and, until this, every mp3 player too, right back to a Diamond Rio PMP300. If it hadn't been for the fire risk, which really prevents me from lending the old thing to someone else, I'd probably have kept it.

Still, the new Nano is an impressive little thing. It weighs very little (it's almost light enough that one could keep it only on the cable), the built-in clip is a nice idea, and the touchscreen works nicely. The build-in pedometer doesn't compare very well with a decent Oregon Scientific one, however - it seems to be rather arbitrary and unresponsive.

The most elegant solution to the calendar I've seen is JRR Tolkien's (yes, him) Shire Calendar:

• It's fully conformant with the astronomical realities (no magical even-divisions or date fudging necessary).
• There are still 12 months (so no weird decimal months, no 34th of Thermidor bollocks). You can stick with the familiar month names (rather than Tolkien's Hobbity ones).
• Each month is 30 days long (simplifying accounting, pay calculations, holiday accrual etc.). No pointless variation, no mnemonics.
• Year on year, a given month always begins with the same day of the week. Even for leap years. So if you were born on a Tuesday, your birthday will always be Tuesday.
• The clever part (which allows all the other stuff to happen) is there is a winter festival holiday (2 days) and a summer festival holiday (3 days normally, 4 in leap years). These aren't week days and aren't in a month - they're special. So e.g. Christmas doesn't change between sometimes being in the weekend, or adjacent to the weekend, or midweek - Christmas is always in the same place. I know I always get disoriented around Christmas - Christmas already seems like a special day which doesn't resemble a Thursday or a Sunday or whatever - the Shire Calendar is just a realistic expression that it's not a weekday, and that it shouldn't be regarded as one. And the first day back at work after Christmas is always a Monday.
• The winter and summer festivals are pretty consonant with common practice in many countries anyway. Move Christmas into the yule holiday (Jesus wasn't born in December anyway, so it's no less Biblically correct than current practice). Many countries have a midsummer festival or summer bank holiday and US independence day can be celebrated then.
• You only need one printed calendar (not the 14 different types we currently need) - you just score off the leap year or not.
• Its easy to fix the locations of other festivals, like Thanksgiving, and then you get a perfectly consistent gap between e.g. Thanksgiving and Christmas
• From a software perspective it's a wash - 2 more mini-months need to be handled, but less bother with differently lengthed months and much easier day-of-the-week calculations.

Adapted from my Slashdot post here

• in the beginning the people meet
• in the middle they argue
• in the end they get together

• in the beginning the people meet
• in the middle they get together
• in the end they get eaten by dinosaurs

I've never competitively run further than the 12km Bay to Breakers, but I'd prepared pretty thoroughly. On the day the various suspect joints that I feared might let me down all performed perfectly, but I messed up a bit on the hydration plan, and an uncharacteristically hot Glasgow left me rather seriously dehydrated by the end (but that's no excuse, particularly for the 5533 people who finished faster than I). I did it in 2 hours 9 minutes, which was about what I estimated beforehand, but I'd have done a deal better had I drunk properly beforehand.

I'm especially grateful to the kind folks from the Glasgow Sikh community, who set up their own unofficial water station (amid an inexplicable five-mile gap between official water stations) and who had some Kärcher pressure washers to douse the overheated runners.

I've been dehydrated running before, but never this badly, so I was in a pretty feeble condition afterwards. It's unpleasant.

A Google Map showing a GPS trace of the route on Google Maps is here (it's weirdly punctuated in the latter section, I guess due to the water spraying).

The race organisers have placed a CSV of all the half-marathon finishers (with splits) in a machine-parsable file here, so I've had some fun grinding through them (it's not like I'll be walking anywhere today).

Of the 8482 runners who finished (they don't give statistics for non-finishers) 5187 (61%) were men and 3295 women. The male winner was Kenyan runner Joseph Birech in an eye-watering 01:01:26, and the female winner was fellow Kenyan Flomena Chepchirchir in 01:09:26 (beating, I believe, her PB).

The mean time for the whole field was 02:03:32, with a peak of people finishing around 01:57 (what, did you guys hold hands?). For men only the mean time was 01:57:20, for women only the mean time was 02:13:17

My own time put me 6 minutes slower than the overall mean, and 12 minutes slower than the male mean. A graph showing the distribution of runners' finishing times is below - the overall mean is the green line, I'm the puffy and out-of-breath red one.

In the last 6k I overtook 176 people but 203 people overtook me (so that's a net 27 people who'd taken better care to drink properly). Of these runner #13180, Angus Denham, went from being 5 minutes behind me to finishing nearly 3 minutes ahead. Perhaps Angus is really Batman.

Of the whole race, the most impressive kick (the fastest last section in relation to their first 15k) was by runner #19069, Gary Clelland (splits 00:34:05, 01:20:14, 02:04:03) who seems to have had a horrible middle race, but recovered to do the last 6k in 31 minutes.

BEEP!

There are two alarms in my house. They're mains wired with a 9V battery backup, and when that battery starts to fail (it seems when it gets below about 7.7V) the alarm beeps once, very briefly. The beep is frustratingly irregular, and there's at least several minutes between beeps. There's no light or other visual indicator on the alarm which is failing. And (as anyone who has heart the siren of an emergency vehicle echoing around the buildings in a densely built up city) it's difficult to spacially locate a high-frequency tone that's echoing off hard surfaces.

BEEP!

The alarms are mounted up high, on the ceiling, they're difficult to remove (push a little tab with a knife while turning the whole thing anticlockwise), and the screwthread is a bit overpainted. There's residual capacitance in an alarm, so even if you remove its battery and disconnect it from the ceiling supply, it will still muster the energy to beep a few more times. I think I know which alarm it is, so I take it down, open it, remove its battery, find a spare 9V, replace that, restore the detector to the ceiling, and go back to bed. Mission accomplished. All is peace and quiet. Sleep.

BEEP!

Darn - I must have changed the wrong one. Find another 9V battery. Remove the second alarm (which is much more intransigent than the first), fix it, replace it. Now they're both done. Sleep.

BEEP!

It's 3:30 and this is impossible. Can I have put a bad replacement into one of them - but which? I take them both down, and find the Fluke DVM. How are "normal" people supposed to solve this kind of problem in the early hours of the morning (where "normal" people don't have a stash of 9V batteries and a digital multimeter) ? I test both "good" batteries - they're showing slightly more than 9V. The two "bad" ones aren't much different - one is 8.3V and the other 8.7V. I don't have any more batteries to try, it's late and I need to work tomorrow, and there's obviously nowhere open that will sell me two 9V batteries at 4am. I can't think. I disconnect both batteries and leave both alarms on the desk. I pile clothes on the alarms to deaden the sound (knowing they'll beep for a while until their caps have discharged), close all the intervening doors, and go to sleep.

BEEP!              BEEP!                          BEEP!      BEEP!       all ... night ... long ...

In the morning, when I feel only a little bit more awake, I still can't figure out the problem. There's still a periodic beep, and it can't be either of the alarms. As I stumble around, another BEEP! And then I find it - a third alarm, this time a carbon monoxide detector, hiding in the cupboard beside the heating system. Its battery is 7.2V, and given the 8.7V one it's happy.

It's stuff like this that makes well-meaning people throw their smoke alarms away, or at least permanently neglect to put batteries in them. I understand why it has to beep, and why it can't flash a permanent light to say it's in trouble (they're trying to maximise the time of protection I get, even at the expense of my sleep). It's a very simple and cost-optimised device, so it's difficult to think of a sensible way for it to communicate. All it can do is beep.

But do all three detectors have to beep the same? The beep function is a simple piezo electric sounder driven by a simple frequency generator (I guess an XTAL) and gated by a transistor. So:

• At the very least it should be straightforward to substitute a different frequency generator for different detectors in a set. A set of detectors might have one at C, one at G, and one at E♭. A detector bought singly could have a pair DIP switch to set one of four codes.
• If a throwaway novelty greetings card can play a tune, is it really too much cost to have the smoke detector tap out its ID code (maybe two digits, each 1-6 beeps, with a gap between them)? The code can be cooked into the detector, with a little screenprinted decal on the side showing its code.
• With only a little more cost, make each detector play two notes concurrently with a fixed interval. Right now I'd hope for a nice consonant interval like a major 3rd or a perfect 5th, but I suppose smoke alarms are supposed to sound horrible. So one detector could play a tritone, one a major seventh, and another a minor second (all with different tonics). Those all sound nasty and urgent, but they also sound different.

The GNU dd has a nice feature whereby it prints its progress if you send it a SIGUSR1 (e.g. kill -USR1 1234). But if the dd process is wrapped in other processes, or isn't attached to a console, then you can't easily get this information.

Say we're running dd if=/dev/zero of=/dev/sdc

How do we figure out how far it's gone?

To the rescue comes the admin's other best friend, Vic Abell's invaluable lsof

lsof -c dd | grep sdc

reports

dd      2662 root    1w   BLK   8,32 0x187e768000    5482 /dev/sdc

The key part of that is 0x187e768000, which is the offset it's reached, in hexadecimal.

To simplify checking that, the following quick and dirty Python script (which takes the filename or device to search for as its only argument) will report the offset field in the lsof output, handily converted to Mb and Gb.

#!/usr/bin/python

import subprocess,sys

if len(sys.argv) !=2:
    print 'usage:n  %s <output file or device>' % sys.argv[0]
    sys.exit(1)

r = subprocess.check_output('lsof -c dd | grep %s' % sys.argv[1], shell=True)
val = r.split()[6]
if val.startswith('0x'): 
    base=16 
else: 
    base=10

v2 = int(val,base)
print '%d Mb (%d Gb)' % (v2/1024**2, v2/1024**3)

So if the python script is called dd_progress.py and we run dd_process.py sdc the output will be:

100327 Mb (97 Gb)

If history was like computer games, the Second World War would have ended when Franklin Roosevelt and Winston Churchill sneaked into Hitler's castle (through a ventilation duct) and repeatedly shot a small glowing area of Hitler's neck with their rocket launchers and bakelite laser guns.

If computer games were like history, a game of Starcraft would consist of years of committee meetings about increasing bauxite production and improving the machine tool lubricating oil supply chain, and you'd get an email once a week giving you a statistical breakdown of the friendly and enemy units destroyed in the conflict.

In the short run they're looking at making a phone that's a few mm thinner than the current ones. But in the longer term they're thinking beyond what we currently call a "phone". They're looking at very small form factor devices which keep their data in the cloud, are configured by another (arbitrary) device which talks to the same cloud, and which make either sporadic or continual data connections with whatever available networks they find, to keep up to date. Imagine very small devices (wristwatches, eyeglasses, earplugs) with 802.11/UMTS/WiMAX radios (which use a mini-sim to identify themselves to whichever network they encounter). And they're thinking about these things as universal identifiers and payment tokens.

Right now you go running with an iPod. Instead you'll have a iPlug, a pair of little in-ear headphones, but with no cable and nothing strapped to your arm. You set up your music program on a tablet, and it seamlessly syncs. You run further than you'd expected, so the iPlug connects to the network and downloads more music. Miles from home your knee gives out. You touch the iPlug and say "taxi". A taxi comes (sent by Apple to the location the iPlug knew; Apple gets a dollar from the taxi fare, which you pay using the iPlug).

You have a iSim unit in your iWatch. You're thirsty, so you touch the watch and say "coffee shop". The watch face shows an arrow to a nearby one, and the distance, and walks you there. Apple gets a dollar. You buy a drink with the iSIM as a payment token (Apple gets 30 cents) and sit down at a table. The table's surface is an active display; it talks to your iWatch and opens a connection to your account in the iCloud. Your personal news appears, your emails, your documents. You do some work, browse some stuff, and when you're done you stand up and the table blinks off. Things will be as you left them when you next peer with an active display - at home, in the car, on the train, at the office, on the beach.

All of this stuff has been done, in various disconnected ways, already. You can pay for stuff with your phone, in some places. Most Europeans (well, Brits at least) have smart cards in their credit cards. You could hotdesk 10 years ago with a SunRay (kinda). You can unlock doors with a Dallas iButton token. Having super-cheap super-light totally ubiquitous networking makes the whole thing join up into a compelling, powerful, system.

You'll never be alone again.

Adapted from my Slashdot posting here. Several posters astutely point out that tiny devices have tiny batteries and so short lives. For a device that's mostly connected to local (low-power) connections, that isn't configured to receive calls (which means it doesn't turn the radio on every few seconds to check for messages or calls) and which you habitually dock to recharge every day, this isn't an unreasonable ask.

To use it you need to plug your guitar into your Linux PC. It's possible to do this with a direct connection but the signal will be weak and noisy. It's much better to use a proper boosted instrument connection - there are many of these, but this posting talks about setting things up with the inexpensive Behringer UGC102 USB device. Various online merchants sell this for around £30.

Once you've got that, the following guide lets you get Rakarrack working in Linux. I've tested these instructions in Ubuntu 11.04 (and it mostly works the same in earlier Ubuntu releases); hopefully things should be much the same in a similar modern Linux like Fedora.

1. In Ubuntu Software Centre (or Synaptic, or with apt-get) install the package rakarrack (which will entail you enabling the "universe" repository, if you haven't done so already). By installing rakarrack you also bring in its dependencies, including JACK audio system and its associated utilities.
2. Save the following script (say to rak.sh) and make sure that script is executable.
   
   #!/bin/bash
   # The following assumes that the Behringer UGC102 is ALSA device #1
   # if it isn't, run aplay -l to see what it is
   
   jackd -R -dalsa -D -Chw:1,0 -n3 &
   sleep 1
   rakarrack &
   sleep 1
   
   # mono Behringer inputs to rakarrack stereo input
   jack_connect system:capture_1 rakarrack:in_1
   jack_connect system:capture_1 rakarrack:in_2
   
   # rakarrack stereo output to both main outputs
   jack_connect rakarrack:out_1 system:playback_1
   jack_connect rakarrack:out_2 system:playback_2
   
   # wait for rakarrack to be closed
   wait %2
   
   # kill JACKd to tidy up
   killall jackd
   
   
   
3. Plug the Behringer device into a USB port and a guitar into it.
4. In Terminal run aplay -l which will list the audio devices that ALSA can see. A typical report will look something like this:
   
   **** List of PLAYBACK Hardware Devices ****
   card 0: Intel [HDA Intel], device 0: ALC883 Analog [ALC883 Analog]
   Subdevices: 1/1
   Subdevice #0: subdevice #0
   card 0: Intel [HDA Intel], device 1: ALC883 Digital [ALC883 Digital]
   Subdevices: 1/1
   Subdevice #0: subdevice #0
   card 1: default [USB Audio CODEC ], device 0: USB Audio [USB Audio]
   Subdevices: 1/1
   Subdevice #0: subdevice #0
   
   
   In this case "card 1" is the Behringer device (it may vary on your system).
5. If the device is reported as being different from "card 1", change the 1 in hw:1,0 in the script to the appropriate number.
6. Run the script.
   In some configurations, PulseAudio has exclusive ownership of the alsa device, and so jackd can't run. In that case, prepend pasuspender -- to the start of the jackd line in the script.
7. Rakarrack will appear. You will need to turn on the effect rack (the top-left button, labelled FX On) and generally you need to raise the input level and maybe the output level.

Merchants that make repeat payments on the same credit card have to keep the customer's credit card info around. This makes breaking into them a tantalising prospect for any attacker, as the merchant stores enough information to allow fresh payments to be made at any merchant, or for fake physical cards to be manufactured. This posting discusses two proposals that allow merchants to make automated repeat payment requests on the same credit card, but without the merchant's permanent database storing that crucially re-usable credit card information.

In the examples below, the card data (CD) is a record containing the cardholder's name and the card's number, cvv, and expiry date.

1. The public-key cryptography method

1. The merchant receives CD from customer
2. The merchant builds a merchant+CD packet by concatenating their merchant ID (mID) with the data in the CD: mCD=mID+name+expiry+number+cvv
3. The merchant encrypts the mCD with the acquirer's public key. EMCD = E_acquirer-pub(mCD)
4. The merchant then forgets the name, number, and cvv fields. They keep the expiry around so they can prompt the customer to renew their card details when the card is close to expiring.
5. The merchant builds their ordinary payment-approval request to the acquirer by concatenating the EMCD with details specific to the transaction (chiefly date and amount) and sends this to the acquirer over the normal encrypted channel.
6. If the acquirer approves the transaction, the merchant retains the EMCD in their database along with the expiry date. For future transactions they build the payment-approval request as above, without rebuilding the EMCD.

B. The payment-token method

1. The merchant receives CD from customer
2. The merchant builds the normal approval transaction and sends this to the acquirer as normal.
3. The acquirer approves the transaction, and as part of the approval returns a repeat-payment-token (RPT), a lengthy random number.
4. The merchant forgets all the CD except the expiry, and stores the expiry and RPT in their database.
5. For subsequent transactions, the merchant builds a repeat-payment-request transaction, and instead of the CD it uses the RPT. On receipt of this, the acquirer checks that the corresponding account is valid (in the normal way) and verifies that the submitting merchant is the same as created the RPT. If the mID mismatches, the payment is declined.

In both schemes the merchant does not retain a permanent record of the customer's credit card data (bar its expiry). So if the merchant's system is compromised, and the attacker gains access to the list of customers, including all the data the company stores on them:

• these cannot be turned back into bare credit card info, as the merchant, and the attacker, don't know the acquirer's private key. So the attacker cannot generate fresh payment transactions with another merchant or manufacture a valid physical card.
• armed with the EMCD/RTP the attacker can create further transactions with this merchant, but as the merchant's stored info is specific to that merchant, it can't me used to generate transactions at another merchant.

The public-key method requires the acquirer and merchant maintain a public-key infrastructure, including key distribution, that they would not otherwise. The payment-token method does not, but requires the acquirer to store payment state for each (merchant,customer) pair.

Neither method protects against the same merchant being fooled into producing unauthorised transations, but it firewalls the damage caused by a single merchant's system being compromised to that merchant.

Once thing is clear: if you're one of the few survivors, nothing more useless than an electric car. At least in the short to medium term, you're going to need a good old fashioned petrol car. You can siphon fuel from other vehicles and from the tanks of petrol stations. If this is a real apocalypse (which means almost everyone is dead) rather than a mere zombie mishap, there is essentially enough fuel for you for the rest of your life.

In the longer term (maybe a generation or two, or much less if there are more survivors) then probably a diesel car adapted to burn biodiesel is the way to go, providing you can get the feedstock for that.

But plugging your Nissan Leaf into a windmill? Only if you live beside a windmill, and don't plan on being 40 miles from it ever again.

• The plug problem
  My flat is rather small. Small enough that I could vacuum the whole place in one go (without moving the plug to another outlet) if the designers had put an outlet in the hall. They haven't, but instead they've littered the outlets around the flat's perimeter (surely that's the least efficient way to cable?); so I need to plug the machine into three different sockets to do one small flat.
• That "pumping" action:
  Watch someone vacuuming (or watch yourself). They pump the machine back and forward across the carpet, covering each section up to a dozen times. Why the pumping? I don't think it's necessary; the vacuum cleaner has brushes that lift the pile and a powerful motor to suck out the dirt - what does the pumping action add? I think it's nothing more than race memory - we pump because our mother's pump, and they because their mothers did. Our /grandmothers/ really had to, because they had mechanical carpet sweepers that /relied/ on the pumping action by the operator. When cleaners gained electrical power, no-one thought to change how they used the machine, so we all still pump.

• Sucked into the Dyson vortex - Are Dysons really worth the money?

  As a vacuum a Dyson is good, that is it sucks good, and doesn't suck (unlike most other vacuums, which merely suck at sucking). It doesn't clog (often) and there isn't a bag to clog. But how much does this matter? For most people, vacuuming isn't that important. I don't vacuum that often (ideally weekly) and the whole job is 10 minutes max; any more and I'd just be wearing out the carpet. Even if your house is bigger than mine, dirtier than mine, and even if you consider yourself to be a member of some cleanliness elite, you're still not doing all that much vacuuming. Does the value-add of the Dyson really justify the six to eightfold premium?
  • Dysons are ugly.
    Normally vacuum cleaners are really neither beautiful nor ugly; they're utilitarian plastic blobs that spend the great majority of their lives hidden in cupboards. That's because no-one cares what they look like, and no-one will pay extra for a beautiful one over its merely functional brother. Indeed, it's likely that it's actually impossible for a vacuum cleaner to be beautiful; perhaps nature has simply decreed that they're confined to the range of beauty between dull and vile. A Dyson is ugly on purpose; it's ugly so that you notice it, and so you're aware that the clever people at Dyson have actually thought about the design, and by extension you'll believe that the clever people at Dyson have thought about all the bits of vacuums that the dullards at Hoover and Electrolux haven't. Dysons are yellow and neon purple, colours designed to draw the eye but not please it; you're unlikely to encounter a Dyson on a foggy mountain road (pray, pray, that you do not), so why is it the colour of a policeman's dayglo jacket? So you say "oh, it's a Dyson". Hideousness pervades Dyson's industrial design philosophy, from the Airblade handdryer that resembles a Cyberman's detatchable genitals to the distended Cthonic eye of the (now cancelled) washer. If this design has a value proposition at all, it's to give you a gaudy means of showing snooty Mrs. Tibbins at number 35 that your carpets are cleaner, your clothes fresher, and your hands drier than hers. It is, in essence, utility bling.
  • Dysons are noisy:
    Dysons are noisy, ostentatiously so, like the Gods fighting the Titans over who gets your carpet cleaner. Dyson TV ads resemble Apple's, where disembodied hands silently maneuver the product at fingertip, gliding effortlessly through tasks previously impossible or unpleasant. The real Dyson experience is more akin a the Harrier jumpjet simulator, where you roar around, scattering dust and kids and frightened pets like yokels awestricken by your Olympian potency. Dysons are, I fear, as deliberately noisy as they are ugly. You paid 400 quid for a cleaner, you so expect one that shakes the very foundations of the Earth. I am woman, hear my Dyson roar.

I have a house full of books, and a storage locker somewhere with many more, some of which I've actually read. I love books, which means I should love book shops too. But I don't love Borders, and I won't mourn its passing.

My "local" Borders is in a hideous retail carpark with a collection of other dead-eyed boxes, where nothing ever happens and there's no reason to visit unless you really want something or truly have nothing to do. It's half an hour drive for me, and half an hour for everyone else.

The shelves tell a sorry story The "new age" section is three times the size of the "science" section (and "science" includes the lightest of popular science, all of maths and engineering, and anything DIY that looked harder than putting up shelves). "Self help" is twice the size, as is "religion" - and "religion" is barely about religion at all, but mostly self-help books that cravenly invoke religion to sell themselves ("Jesus wants to you be thin"). When there's more books about the Christian way to stop smoking than there are bibles, and no sign of Aquinas or Boethius or Luther or St.Teresa, it's another sign that they're beyond redemption (so to speak).

Eight quid for a 16 page kids book? A huge collection of 2009 calendars with newspaper cartoons about slippers? Discount books that don't compete with Bargain Books who pay a tenth what Borders do for retail space? Old DVDs with prices that seem designed to compete with Virgin Megastores (look how well they did) - £9 for Casablanca? £11 for Fast Times at Ridgemont High? £12 for some thing with Jack Black in it?

Front of house, their pride and joy, Borders' strategic reserve of unsold crap. Ghostwritten celebrity junk: all called something like "My Life" or "My Way", supposedly written by someone whose life has barely started and whose only achievement is being on some program on Channel Five where they were the outstandingly stupid person among a group of troglodytes carefully chosen by professional psychologists as being the worst possible. TV tie-ins. Celebrity recipies. Celebrity detox. Celebrity diets. Celebrity confessions. Contrite celebrity "when I got out of jail all the money was gone" sob stories. Things one guy from Top Gear likes, and things the other guy from Top Gear hates. I am Jack's raging bile duct. And I swear the prices for this junk have gone up.

People will tell you that Borders, who happily crushed the smaller bookseller when times were good, has in turn been crushed between Tesco and Amazon. This is true, but Borders went along with it. It's no surprise that Tesco can shovel junk better and cheaper than anyone, or that prime retail space isn't best used for selling off old calendars and discount books. With its convenience and range and prices, Amazon is darn hard to compete with, but Borders didn't innovate and really didn't look like they were trying. Relying on the "Grandma doesn't buy books off the internet" theory was a loser for them, because Grandma turned out to be smarter than they thought.

Still, the venture capitalists who own them will have plenty of "Chicken Soup" books to read inbetween insolvency meetings.

A while ago I started rewriting in python, still on cygwin. I really don't know why. It was worth it, I suppose, but more complicated than I'd hoped. In a final push yesterday and today I finally finished it, and cut its dependency on external utilities that cygwin provides (sed and find and stuff). So now it runs natively on Windows or Linux, which is a relief. Cygwin is fine as long as you stay inside the little cygwin box - but if you have to call in and out of it then you run into endless little filename-escaping issues.

The python version is certainly more verbose than its bash counterpart, but much easier to read (and, I hope, to maintain in future). Image scaling is now done by PIL rather than imagemagick (I just couldn't get PIL to work properly on cygwin's python). Where I used hacky sed calls to do some text substitutions I now do some really botched python regexps instead (I'm more ashamed of them that I was of the sed ones, which is saying quite a lot). The whole thing has about as many lines of code (although far fewer backslashes) and takes about the same time to run. But for exactly one line (a shell variable expansion in code called inside the macro system) it's entirely portable (damn, I'd forgotten that hideous DOS %var% syntax). I still use htmlpp to do template expansion, html tidy to fix any html snafus, and linklint to verify that there aren't any bad links. Blogs are still built in an overly manual way, and there's still no RSS syndication.

I keep wondering whether to shift the blogging support to Wordpress (which I've goofed around with, and which couldn't be easier). I guess I'd put the Wordpress content into an iframe (where the current static blog appears now) as I don't really want the entire content served from Wordpress. I'm squeamish about dynamically generated content when static would do, so I guess I'll have to figure out how to get it to produce static content. Urgh, I guess that's another hundred lines into the buildscript.

In checking through my webserver logs I've discovered that several teenager use my images for the backgrounds to their weblogs. Technically it would be better if they downloaded the images to their own website (instead of using up my paid-for bandwidth), but they seem like sweet people, so I'm not going to hassle them about it. If I was particularly nasty I could write some code that checks the referrer on requests for those images, and send visitors to their site (but not mine) something horrible like the goatse man.

It's a compliment, I guess, so it'd be churlish to complain. The thing about copyright is that if I fail to defend it, someone might claim that I've abrogated my right to do so in future. The last thing I'm going to do is send a nasty cease-and-desist letter to these folks, so here's the smart solution - the following myspace users are hereby granted licence to use images from this website on their personal weblogs:

• So Nostalgic
• Deevo
• Lalala LiSa
• Nothing Left
• Ramsses
• crystal
• Charlotte
• Claire
• Ashley

• fuschia2
• godetia
• surfinia
• lavetera
• lobelia1
• lobelia2
• lily
• osteospernum
• pelargonium1
• pelargonium2
• petunia

This was followed by snowboarding in Lake Tahoe. I really can't snowboard, and I do it so infrequently that I forget almost everything in the two or three years between sessions. So this time I squished in a rib (opinions differ as to whether it's technically cracked; it sure hurts enough), twisted an elbow in an unfortunate (but not serious) direction, and probably did my liver irrevocable damage with all that endless apres-ski. Still, I had a lot of fun - if only Scottish ski resorts were as fancy as Squaw Valley and Heavenly. If only Scottish ski resorts had more (like any) snow, and more (like any) sunshine. Perhaps I should invent "rainboarding"?

Here are some photos:

• The Carson valley in Nevada at dawn
• An ancient (?) rusted tin in the Mojave Desert south of Death Valley

After that back to the bay area for a couple of days, including a fancy meal at a faux-sixties place in San Francisco (oh, fine dining is wasted on me) and then off on the longest drive of my life. From Oakland to Barstow (boy, that's not a very interesting place), then a day and a half in Death Valley (sleeping in Beatty, NV, as the accomodation in the valley is unpleasantly expensive). Death Valley blew me away: I've never been anywhere so big, so bleak, and so empty. In the middle of this desolation the little desert flowers were blooming (this, apparently, is one of the best seasons for them in years). It's strange - the smell convinces you that you're at the seaside and it's just low tide. There are salty tide pools and the landform feels just like the seaside. It took more than a day before I could persuade myself that the tide really wasn't coming back in. It would be a great place for backpacking or biking (in the winter; I don't fancy being there in the summer at all). I also visited Scotty's Castle, which wasn't really very interesting.

More photos:

• Golden Evening Primrose blooms on otherwise barren slopes
• Death valley viewed from an ash rim near Little Hebe Crater
• A wet playa lake near Badwater
• Ubehebe Crater, an extinct volcano
• Closeup of some dry mud flats at the southern end the valley
• Golden Canyon
• More exciting dry mud

From Vegas I drove west to Flagstaff and then north over Coconino (at night, in a fair amount of snow) to Tusayan. The following morning I help an English guy at the gas station, who can't figure out how to pump gas. The first time I did this in the US (a long time ago now) I got similarly confused. In Britain you pump then pay, but in the US it's the other way around - which means if you're silly enough to pay in cash you have to guess how much, or overpay and then go back for change. Plus there's the pointless handle you have to lift to make the pump work (I've never found any need for that).

The vistor stuff at the south rim is way touristy, and one is left walking a tame little path with hundreds of bickering teenagers. I'm a bit underwhelmed, but that's probably entirely a function of only being at the tame top, and not having time to venture down into the canyon itself.

Photos:

• One of the fancifully named "temples" on the north rim
• View westward
• Snow on the south rim

Monument Valley is big, and impressive, and certainly worth the (rather hefty) trip. Like (almost) everwhere else on this trip, I wish I had a lot more time to explore properly, and to do so on foot. Even from the tame road around the easy bit, it's an incredible place.

You get that same uneasy feeling you do in any inhabited touristy place - the people there want your money, but they really don't want you. There are signs asking you not to take photos of people's homes (or, rather, signs saying you'll have to pay to do so). I don't take any; doing so would seem to require rather too much of a condescending "oh look at the quaint little houses the poor people live in" attitude. This is the poorest place I've been in the US, and it's noteworthy how many "Support Our Troops" signs there are here; I don't know what else folks would do for work.

Here's four photos of the valley. Even after pouring over the USGS geodetic map I still can't figure out which mesa is which, so if someone can help me with that, I'd grateful:

• 1
• 2
• 3
• 4

I didn't go to Canyon de Chelly, largely due to not bothering to read the next few pages in my guidebook. I did briefly stray into Utah (my first time) after I missed the Monument Valley turnoff, and I'm glad (and slightly surprised) to report that I didn't instantly turn into a pillar of salt. Worse, I didn't leave time for Arizona's meteor crater (something I only noticed on the map once I'd gotten to Phoenix).

From there it's a long drive south to Flagstaff and then down into Phoenix. I didn't have much time in Phoenix (I didn't arrive until way after nightfall) so I can't say much about it. Well, other than everyone drives very fast and the highway numbering is crazy.

Now, one question I need to address is the sat-nav issue. My Dad has a TomTomGO unit, which he's very impressed with (I less so, having been sent some weird roads in southern Glasgow by it). I could have spent £100 on the US mapset for it, and I probably wouldn't have gotten lost. I did get lost, but not drastically. I can never navigate around suburban east Oakland, but the Tom Tom sometimes lets you down in such places. Flagstaff's freeway signage confused me a fair amount, and Phoenix was terrible. Vegas was easy, and every other town I was in only had one street. So I'm on the fence - I may have wasted two or three hours over the whole trip driving on the wrong road; on the upside, I didn't have to worry about whether to leave that expensive satnav unit in the car overnight.

I've you've not been, I couldn't recommend it enough. Beautiful, clean, friendly people, nice architecture, lots of interesting stuff to do. The food was great - I don't know how I'm going to go back to eating my own cooking after living on delicious, cheap, and varied tapas. Like all "ethnic" food, the less you pay for it (and the plainer the place you buy it) the better it is. We ate in one place under the Montjuic which served us each a fish, three slices of tortilla espanola each, and two (eerily British) desserts. That and a bottle of water and a bottle of Estrella beer (the latter for me, naturally), and the total cost came to eleven euros. Heck, you couldn't eat from McDonalds for that (and they barbarically don't serve beer).

How hard can a Spanish omlette really be? (I may have to consult Delia on that one, hopefully she does more stuff than just Yorkshire pudding). Naturally I took a bunch of pictures (it's so nice to be somewhere where there's enough light for cameras to actually work properly).

• segrada1
• segrada2
• batllo1
• batllo2
• batllo3
• calatrava
• graffito

This post is adapted from an anonymous post I made on Slashdot earlier

To clarify, Neutrino is the (current) OS and QNX is the company (to confuse things, QNX used to make an x86-only OS called "QNX" or "QNX-OS", which is quite similar to, but not the same as, the multi-architecture neutrino).

I have some experience of both programming for Neutrino and some business-development work on projects aiming to deploy neutrino. I have both very positive and rather negative things to report.

On the upside, the Neutrino OS is generally excellent. It's very responsive (from a real time perspective) and the system and device APIs are nice and clean, pleasantly symmetric, and well thought out. Writing device drivers is a much more pleasant business than it is on Linux or Windows. The microkernel stuff really isn't visible to a user, but it makes the low-level developer's life a deal easier. There's a great satisfaction to recompiling a video driver, slaying the current instance and executing a fresh one, and have the whole thing work without a reboot.

Photon is okay. It's fast but rather old-fashioned, and its C API is crufty and rather a pain to code in. It's rather thin on higher-functionality widgets and one has to do more heavy lifting when implementing one's own widgets that I'm used to. It doesn't have a more modern graphics API (like GDI+/quartz/java2d) and that's a bit limiting when one intends to use it for TV/Video stuff (settop boxes etc.); again, I can do it myself, but it's more heavy lifting than I'd expect on other OSes. Support for audio and media is so-so, and I don't believe there's any 3d support. None of this is a problem if you view Neutrino as a high-end embedded OS (as opposed to a desktop OS) but even there - I'd rather not implement a nice post-Tivo setop UI or a high-end incar navigation system on Neutrino - it's all doable, but its rather too much work. Photon is clearly architected for speed and real-timey-ness (it's single threaded, like Swing, but being in C one doesn't have access to some nice little things that make Swing programming more tractable, like invokeLater), rather at the expense of programmer friendliness. One has to ask, however, if it's really worth the time of the idle user learning photon, and the low number of free and open source (and heck, commercial) programs using it shows that most developers haven't learned it. There is an X-server for neutrino, but I really don't know anything about it or the degree of toolkit support on it.

The real problem with Neutrino is (or was, maybe things will change under the new regime) QNX (the company) and their business model for selling neutrino. It's not that they're dumb or mean guys, but things conspire to make the independent developer's life (and particularly the free/open-source developer's life) discouraging. Here's some of the problems I faced:

• Neutrino is closed source, and in practice much more closed than say Solaris on Windows. There's a lot less example code floating around, either from official sources or other developers, and QNX's documentation is mostly "what this call does" not "how to do XXX".
• QNX's support (the paid, and not at all cheap, support) isn't all that good, and several times has seemed like a branch of the sales department - a couple of times I asked for API docs for module interfaces that clearly existed (as QNX insiders had written a bunch of plugins of one kind or another) and was told I could (for a significant price) licence the source code to the particular element. That's no good - first off I don't really think I should have to pay (again) for just an API, and secondly I really don't want to be reading through some code to answer my question.
• QNX is a pretty small company, and its engineers have their hands full. So device support is limited and rather behind-the-times. Folks have a view of embedded systems that they're little boxes like Audrey, but a firewall or an airport-information-system or a storage array can be a pretty big-ass piece of iron, and it's reasonable to expect to use some new, high-end cards. Bar some drivers ported over from xfree86, QNX's answer seems to be "our custom-engineering department will write it for you". Again, I understand why things are this way, but that doesn't make it encouraging, and when I can turn around and get a free or low-cost driver for windows or linux or (increasingly) for VxWorks, it makes choosing neutrino harder.
• Community and print resources are poor. There are essentially no print books about programming neutrino, other than a rather scanty book that QNX give you (when you buy the rather expensive professional momentics dev package). Yeah, there is one (core-OS centric) programming Neutrino book, but it's out of print (amazon couldn't find me a used copy after months of trying). All the other stuff (networking, graphics, photon, media, device-specific stuff) isn't covered at all. Similarly newsgroups and mailing lists are very limited.
• And lastly (this is just a gripe, really, but I think it's indicative of QNX's attitude in general) is the FUCKING licence manager on the windows-hosted platform for momentics. You pay a ton for a single developer seat and have to waste days diagnosting problems with the stupid portions of the system not taking to the dumbass licence manager, or its being misconfigured in some silly, hard to find, no-diagnostics way. Support is largely clueless about this. It's all the more frustrating because the toolchain works without licence-bollocks when hosted on a neutrino machine (great, so I need another machine in my teeny office) and they're mostly protecting a version of the GCC toolchain and the eclipse IDE.

So Neutrino is pretty good in its little embedded/control space. It has great potential to be much more, but I can't see how it'll get out of its current space. Just as Linux benefits from a virtuous circle of support, features, and acceptance, Neutrino suffers from a vicious one. Why code for it when there's so few users and the tools and docs are second-rate? Why improve the tools and docs when there are so few programmers. Why try to help expand the neutrino community when QNX aren't really terribly motivated to help you.

It's been clear for a while that QNX (the company) have been under financial pressure. I guess they're getting squeezed on one side by VxWorks and on the other by Linux. I'd hoped they'd find a way out of their niche (perhaps by open-sourcing some core stuff, perhaps some other means) but their being a remote division of a speaker company surely won't make that likely.

I conclusion: nice (core) technology, but a business model that hasn't kept pace with the times.

Update (years later): For context, "Audrey" in the post above is the old 3Com Audrey internet appliance.

• autumn
• leaf2
• web

They're all of flowers, taken in the garden of a friend of mine. You know, I'm really not particularly a fan of flowers in general, but if you want to take a beautiful photo quickly and cheaply, it's really hard to get a better and more convenient source that some little flower growing in an unused patch of land.

Also, by popular demand (not least from me) I've chopped the photos page into sub-pages, sorted "thematically". So, while there's still an overall index page, things are also grouped into more easily downloadable sections.

Wiki is great for the task for which it was intended. It's great for collaborative editing of text. To date it's pretty much useless for collaborative editing of anything else (there's no reason once couldn't have a graphics wiki or a midi wiki, for example, but no-one has written the tools to do this).
Wiki isn't great for conversation, isn't great for process (which is workflow-managemant, I suppose) and really isn't good at all for voting.

• Archiving is hard - pages grow to large sizes, so someone deletes bits or moves bits or archives the whole thing in some repository and the discussion starts anew. This all has to be done manually, and it destroyes the flow of the discussions. As there's no structure to the discussion, there are no logical points at which to perform such an archiving procedure, and there are no clues to allow either people or software to later reconstruct the stream of what was going on.
• You can't permalink a comment or a discussion-tree. If it gets altered or deleted or archived, it's essentially lost (sure, it's in there somewhere, but in a form that means you have to go searching for it.
• It's almost impossible to syndicate and aggregate. There's no "we're done now" event in wiki, beyond the simplest edit-update. As edits can (and frequently do) go back and touch previous content (inserting comments, fixing bad links, reformatting, etc.) a naive RSS channel of a single wikipage would be filled with trivial update edits (and with no way of telling the client to glue all the little changes together into a single object).
• There is no structure to conversations, other than arbitrary conventions (often formatting-based). So it can be hard to follow a conversation, hard to see how an old discussion got into the present state, and there are no cues to allow software tools to figure out what's going on. So there's no way to automatically filter (in or out) a discussion or a thread that's happening on a wikipage which has multiple things going on simultaneously.
• The very plasticity of the wiki medium means that one person's comments are subject to infinite alteration by others. Although serious cases of this tend to be quite rare, one has to be a skilled operator of the change-history system of the particular wiki to figure out if, when, and by whom a (sometimes very subtle) change has been introduced. With those wiki systems which don't have history (or don't present it publiclly in a very useful way) then it can be almost impossible. Votes can be tampered with, and conversations altered to make the guilty appear innocent and the innocent guilty. Worse, if a discussion is subsequently archived (using the common cut'n'paste archival method) then the history is essentially lost.
• Voting is cumbersome, needs manual intervention, and is susceptible to fraud. Wikis often need to vote (what to delete, promote, laud, fix, move, rename, split, join, and what general policies and proceedures need to be adopted). Votes are conducted as general wiki conversations, and have all the problems associated with those, and some special ones of their own.
  • There's no means of checking that a given voter is qualified, and has voted only once. (this has to be done manually, something that doesn't scale well)
  • Managing more complex voting systems (such as transferrable voting) makes for more manual labour.
  • Discussions get muddled with votes, and untangling the two can be a tiresome (and yet again, entirely manual) business.
  • As with all wiki discussions, there's room for deliberate vote-tampering (by changing or manufacturing votes by others, or deleting unfavourable votes altogether). There's also room for entirely innocent accidental vote alteration or deletion.
• There are no hooks from which to hang a decent workflow system.

  Items need to be created, edited, checked, cleared, approved, and published. This isn't so much of a issue for an edit-forever public wiki like Wikitravel or Memory Alpha, but wikis which produce works in a finite time (such as those in professional organisations) may find the discussion of work and the actual work done itself dislocated. Resolution of discussions and concomitant changes in the work items must be done manually.

  To be fair, wikis weren't really built with workflow management and automation in mind, and don't have support for it anywhere, not just in the discussion pages.

1. The forum needs to have proper permalink and thread handling, and its interface should be able to present these sensibly (with permalinks obvious and with the user being able to open and close the "treeview" of a given thread or threads.
2. As this is a discussion about the wiki or its contents, users must be able to embed wikimarkup (including wikilinks, images, tables, templates, etc.) into forum items.
3. As there still all advantages to collaborative editing of "meta information" such as discussions, the system either shouldn't replace talk pages entirely or (better, I think) forum entries can have a "is editable" field (which can be changed only by the original creator of that item). Editable items behave exactly like miniature wikipages (and if the permalink code works properly, should be able to stand alone as templates or macros).
4. The forum should be integrated with the wiki's user account management system, including single sign-on and support for common policies (administrators, banning, flood prevention).
5. Integration with the wiki software's UI, including common stylesheet elements and proper cross referencing between the discussion and the article interfaces.

• basking, a photo of some seals (harbour seals, perhaps?) basking on a sandbank on the Pacific coast of Sonoma County, California, just where the Russian River flows into the ocean.
• sonoma_coast, some of the amazing rock formations that dot the Sonoma County coast. These are also from the Russian River area, taken rather early in the morning (hence the unfortunate sea mist). I saw some similar formations further up the coast (near Elk) later in the day, but by that point I was late for an appointment in Oakland (which is far from Elk) so had no time to stop.
• golden_gate, a shot of the Golden Gate bridge taken from the Marin side (following, incidentally, my getting horribly lost in the Presidio while driving a gigantic rental truck). You can see parts of San Francisco, particularly that TV transmitter tower (Sutro Tower, I believe it's called).
• breakers, a shot of Pacific waves breaking on the shore of Manchester State Beach, just north of Point Arena. Man, I stood in that damn ocean for ages, alternately freezing my feet off in the icey water or burning them on the blazing sand.
• shimmer, a simple texture shot of the receeding water, again near Point Arena.
• foam, another texture shot, showing that frigid oceanwater. I'll spare y'all the numerous photos of my bare feet turning blue.
• driftwood, a shot of an interesting structure someone built from driftood on Manchester State Beach. Visible in the background is the Point Arena lighthouse, which is (apparently) the tallest on the west coast of the US. You'll notice I've managed to entirely avoid that tired photo cliche, where we try to draw some lame allegory where the driftwood symbolises death or corruption or something. Nope, it's just a pile of wood on a beach. Sorry.

In the meantime, I've moved mcwalter.org and mcwalter.net from their prior home at plugsocket.com. There's nothing particularly wrong with Plugsocket (other than they're maybe rather overpriced) but they don't offer java or python and their config panel is the aging Sun Cobalt thing (which is fine, but horribly limited). So we're moving (or have moved, hopefully) to javaservlethosting.com. The DNS update has taken here, so I see the new version, and it seems to work fine, and at least seems to be a bit faster than before.

In a fit of getting stuff done, I've added some background images that have been sitting around waiting to be put up for ages. There's the three:

• sky, an attractive sunset taken from Stirling's King's Park last month
• usbkey, a supposedly artistic closeup photograph of my little USB flash memory key
• lily, a cute (in a blurry kind of way) photo of a nice spotted lily. This one was taken in someone's garden in California. No, really.

The cherry trees are in blossom now, and the japanese maple tree seems to have grown a whole new set of foliage virtually overnight. So for this month or so my little part of Scotland looks, to some extent at least, like Kyoto.

Of the usual myriad of sub-mapplethorpian flower photos I've only added the following three:

• blossom1
• blossom2
• bluebells

• camellia1
• camellia2
• doggus
• duckling
• rabbit
• ripple2
• wettie
• daffodil_3 (yes, another darn daffodil - they're pretty, dammit!)

More exitingly (?), I've also made a new section, presenting images suitable for you to download and use as the background on your PC. Frankly, they're not really all that good, so I doubt they'll actually compete with that one of Halle Berry's boobs that you're already using, but I can life in hope. Take a look a the backgrounds section. Your feedback, including technical feedback, is most welcome.

I've taken a bunch of my trademark (hackneyed?) low-level photos of daffodils growing in on the lawn of the HQ of Central Scotland Police. I'm quite glad they didn't take ill to my standing around on their lawn taking photos, all too many of which have images of their (rather unattractive) buildings, antennas etc.

The two photos I've added to the image page are:

• daffodil_1 (that's a shot of the sun shining through the petals)
• daffodil_2

If it's not clear from the photos what they were doing (yes, it really isn't) the workers had long (maybe three metres) wires with some kind of fuel-filled rags tied to the far end. They walked back and forward across the field, leaving a (rather unimpressive, frankly) trail of fire behind them. If the flames appear at all scary in the shots then that's entirely a function of my being about six inches away, with the camera right down on the ground.

But spammers are professional, they're technical (or rather, those who write spamware are), and they have a strong impetus (profit) to continue in their line of work. Their goal isn't to send email, per se - the want user impressions, and they don't really care how they get them. Hitherto email and newsnet spam has been the easiest way, and so they've mostly stayed there. With email becoming a less hospitable environment, they're going to move elsewhere. They already are.

BBC news reports "spammers are targeting blogs". Equally, instant messenger and chat spam is now commonplace. This isn't a diversion - this will be the new battleground for junk postings. It'll follow the same trajectory as it did for mail, and it will be just as hard to stop.

Any website which allows users (anonymous or registered) to make changes to the website which other visitors can see will be used. Some examples include:

• weblogs (both those managed on professional websites like Blogger and those managed on individual hosts.
• wiki based sites, such as wikipedia.org
• community groups such as slashdot.org and plastic.com
• open-source software sites, which allow posters to post messages, file bugs, and in some cases upload software changes. Examples include bugzilla.mozilla.org, savannah.gnu.org, and sourceforge.net

1. Insert their (textual) spam copy directly into pages
2. Insert graphics, applets, flash, or other media directly into pages
3. Insert DHTML code that pops up windows directing visitors to their own content
4. Add links to their own content, in the guise of ligitimate content
5. (in media where alterations can be made, such as wiki) subtly alter existing internal and external links to point instead to the spammer's content

• Where anonymous changes are possible, that's what they'll use
• Where access is uncontrolled but IP addresses are public, they'll keep as large a pool of addresses as they can, and use each address on as many sites as possible (a horizontal distribution). This makes a single site's blocking an existing address as ineffective as possible. Rather than flood, some of these accounts will perform incremental changes in a piecemeal fashion. A block of text or a URL will be tranformed from valid to spam not in one go but by steps, where each step doesn't make editors or reviewers particularly suspicious.
• Where registration is mandatory, they'll register using throwaway email addresses (they've already got plenty of those). For a while (until they're blocked) they'll have multiple accounts per site per email address. They'll setup SMTP servers on dialup accounts to handle these.
• They'll create sophisticated, site-specific scripts to make their changes. The more heavily policed a site is, the more clever these scripts will become.
• Where sites are policed (and obvious spam quickly removed) they'll have their scripts cut valid content from other pages or posts on the site, make some (less than obvious) spam changes, and repost this as a new entry, page, or post on that same site. This will, in general, produce content that will pass a cursory screen by overworked editors.
• One of the goals of the spammer is to have their spam posting persist for as long as possible. Ideally it'll be picked up by a search engine, and wholesale spamming of various groups and sites will be one way in which spammers attempt to elevate their google pagerank.

• both to protect against HTML/DHTML spam and for security reasons, sites need to be very dillignet about escaping or removing html tags
• anonymous changes are problematic, and will be the most frequently abused. Some sites will disallow anonymous access altogether
• email sites already have countermeasures present to prevent new accounts being created be scripts - these generally rely on asking the client to do something that humans find easy and machines hard (such as recognising text on a noisy field). Spammers have ways around these, but this slows them down. Websites will encorporate similar countermeasures.
• blacklists of IP addresses and email accounts can be distributed between sites (via some kind of clearing-house facility). This is more useful for "webspam" (which is persistent) than for email spam (in which cases the spammer has already abandoned the address before it gets blacklisted). For webspam, once an address is identified as bad, all subscribing sites can flag or remove all postings that match it.

Spam filters put a darwinian pressure on spam and spammers, either to adapt to more effective spam, or to move to a new area where it's easier to operate. Similarly, spam in new venues will put pressure on software for communities, chat, wikis or blogs to adapt or be swamped (as email has) to the verge of being useless. In the meantime, get ready for a web that sucks (in places) as much as email does now.

This photograph, which I took on a horribly hot and humid day this summer. It shows the lower summit of the hill called Dumyat, which overlooks Stirling in Scotland. Down there in the bluisish murky haze one can just see the Abbey Craig, which is topped by the monument to William Wallace.

That purple stuff is patches of heather. It really does grow in that weird pattern, and it really is that colour (infact, it's far more vibrant, but my camera is rather too poor at capturing colour to properly display what it really looks like).

This is probably going to be the last image update for a while (unless I dredge something up from my rather extensive image archive). I have to confess to having broken my trusty Canon digital camera (note to self: don't leave expensive things on the roof of the car). It actually doesn't look too badly damaged (and perhaps needs just some kind of camera-orthopedics), but it'll need fixed.

For the geogeeks among you, it's on the Sheriffmuir above Stirling in Scotland, pretty close to Dunblane. The view is from Sheriffmuir road looking roughly North West, over the valley to the Trossachs. Right beside the cellphone mast, and a large and depressingly modern farm (farmers just don't want to pander to the rustic stereotypes of we jaded urbanites). It's also surrounded by some blackface sheep, which manged to perpetually be in the way and almost never in a cute photographic pose - the only decent photo I have of any of them is a "L.A. style" driveby photo, with only half of a rather angry looking ewe in it.

I have at least a hundred photos of the same area, all taken on the same evening. It's perhaps the curse of using a camera with automatic exposure control that each picture either shows the amazing sky or shows details of the dark landscape. This falls from the latter catagory - the sky was infact infinitely more grand that it appears here.

Sorry, mountain fans, I've really no idea which mountains these are - they're somewhere on the road between Aviemore and Pitlochry (but then, there's lots of mountains between those two places, so that probably doesn't help much). Again, this is a product of the same holiday as yesterday's seal pictures.

Aviemore has a rather bad reputation in Scotland (and particularly in the acid pages of certain guidebooks), as it's long been a concrete ski dormitory. It's really not that bad (having, apparently, been tidied up over the past few years). It's still trying desperately to be a "proper" skitown, but it's got a long way to go before it's Aspen or Zermatt, although it feels just a teensy bit like Tahoe City (which isn't all that impressive, but still). Frankly, after endless one-street Scottish towns and villages which feature only a drab tearoom and a "heritage centre" (note: "heritage centre" appears to be code for "shop"), Aviemore is a breath of fresh air.

Some of our time was spent at the Scottish Sea Life Sanctuary on Loch Creran near Oban. It's a great place ; don't let that ugly-ass website fool you - they're much better at fish and seals and stuff than they are at website design.

The seal pups there messed around with us terribly, hanging around looking as cute as possible and then zooming off into the murk as we fumbled vainly for our cameras. I did manage to capture a couple of decent images (and over a hundred rotten ones). I've uploaded the decent ones in the 'animals' section of the photo page (warning: nauseating cuteness ahead).

Apart from the legal problems associated with the format, GIF is technically outmoded, lacks decent colour imaging, has no alpha support, and features rather poor compression. Ideally it would have been entirely superceeded now either by the vastly superior raster formats PNG and MNG or (for maps, diagrams and other such "drawn" things) by the scalable vector graphics format. PNG support is essentially universal in modern browsers, and both MNG and SVG are bubbling under nicely.

While the burn all gifs page flatly recommends "by switching your site to PNG, you encourage users to upgrade to PNG-capable browsers", this rather stern prescription leaves all too many innocents in its wake - there's still enough users who're stuck (for technical or institutional reasons) in GIF-only browsers. Wise web developers (and me too) believe, rather, in making pages that "degrade gracefully". This leaves web page developers with something of a dilemma - either stick with the antique format yet again, or leave the poor innocents who can't upgrade with a broken website. Whoever said HTML was portable?

It's possible (although tricky and rather non-portable) to pick up image information from a CSS stylesheet, and it's possible (although tricky, very non-portable, and frequently disabled) to use DHTML (i.e. JavaScript and some DOM other other) to alter images according to browser capabilities. Lastly one can dodge the client issue altogether and generate the appropriate webpage on the fly in the webserver - but using expensive server resources is a rather expensive and troublesome solution, particularly to a problem that should be trivial to solve on the client.

What we really need is a fix to the markup that defines images, which presently looks something like this:

Naively, one could fix the problem incrementally (for some amazing new 2009 era file format "FUF"):

This approach (or something more general, ideally with better thought-out syntax) also solves that perennial browser-author issue "what's the point in our supporting it before IE does". That seems to be a significant part of the "do we include MNG" argument that the Mozilla folks are going through right now (see mozilla bug 18574).

Better yet, the page degrades gracefully, so it should be much more accessible for visitors with screen readers and text-only browsers.

I really wanted to change from using a table layout to an entirely CSS layout. Written properly, a CSS layed out page can degrade much better than the usual table-based solution that's the current state of the internet. This would provide a simple and usable page for disabled users and those using limited access devices (like TV set-tops and cellphones). The villain of the piece is, as always, Microsoft. IE6's handling of percentages in CSS layouts differs from the other modern browsers (Netscape/Mozilla, Konqueror/Safari, and Opera). I'm being generous here - really IE6's CSS layout code is broken. I'd love to either not cater for visitors for IE6, or at least give them a crappy experience, but the grim fact of the matter is that around 90% of my visitors (poor misguided innocents that they are) still use either IE6 or or (jeepers) IE5.5.

So I have a plea for moderately technical visitors who're still running Microsoft Internet Explorer - web authors and sad disabled children the world over beg you to try another browser.

For mac users, try Safari or Mozilla.

For windows users, try Mozilla or Opera

Better yet, all of these browsers are free to download and they're all better than Internet Explorer, faster than Internet Explorer, and more standards compliant than Internet Explorer. Users of these browsers also have a lowered rate of gonadic atrophication. Honest.

[ perhaps I should explain: "mozilla" is exactly the same as the Netscape browser, just with Netscape's advertising stuff removed ]

It was previously hosted by Yahoo! in the sunny Silicon Valley. For both technical and logistical reasons I've moved it to Plugsocket Internet, so this page is now being served from the dank East Anglian fen in the low eastern part of southern England. If you remember that "dead marshes" place in the second Lord of the Rings movie? East Anglia's just like that.

Perhaps it's my imagination, but things seem to run much faster from the new location. If you're looking for a decent low-cost web hosting solution, and you don't need the handholding that providers like Yahoo! provide, I can heartily recommend Plugsocket.