Make a Folder Manifest for XML Files

Posted on Wednesday, March 13, 2013 by Jason Coleman

One task that has come up quite a lot as I’m working with a lot of XML files (mostly DITA content) is I need a way to create a list of all the XML files within a folder. More than not, I want this list to be an XML file, too. There’s really no folder- (or even file-) level operations in XSLT to do this. It’s simply not what that language is used for. To do this, I had to create a simple script. Using scrips like this is very easy to integrate into the DITA-OT (though not where I use this particular script).

If you’re a web developer, there’s probably many better ways to go about doing this than using a Windows Batch file. You probably already know many of them. This isn’t intended to be used in a web data scenario, but more for local XML data management tasks.

The Windows Batch File

I personally really like the Windows batch file command language. It’s pretty simple, even though it does lack a lot of nice features¹. When you want to do folder or file operations in Windows, I think it’s the easiest thing to use even when you’re a really poor programming like I am.

This batch file writes three pieces of information to an external XML file:

It writes a root node to an XML file. It also adds the folder path into an attribute of the root node, which can be useful for post-processing.
For every XML file in the folder, it adds a child node after the root node’s open tag. These child nodes will contain a link to these XML files in the folder.
It writes a close tag for the root note.

I refer to this new XML file as a manifest, as it lists all fo the contents (well, XML files in this case, anyway) in the folder. Once an XML file is created with this information, XSLT can then be used to use or change the information in those files by running against this manifest file.

So, MakeManifest.bat looks like this:


SET output=manifest.xml
ECHO ^<manifest sourcepath="%~dp0"^> > %output%
FOR %%f in ("*.xml") DO (
    ECHO      ^<file href="%%~nf.xml"/^> >> %output%
)
ECHO ^</manifest^> >> %output%

Copy those lines into a plain text editor and save it with the file extension .bat and give it a try!. That’s all there is to it. If none of that makes any sense to you, I’ll refer you to SS64′s CMD reference page.

It is worth noting that (and the sharp reader might have figured this out already) this list will include a refernce to itself, itself being another XML file in the folder. You could simply rename the output file extension to something else (.txt, .manifest, etc.), which is a good reason I put in a variable to make that easy to do. It doesn’t affect what’s in the file.

Post-Processing the Manifest File

In my case, these XML files tend to be DITA topics. What I’m really after here is to create a DITA map. With a little XSLT file to process this manifest —which can be run from the same Windows batch file— it’s easy to create a DTIA map for all of the DITA topics the script finds in the folder.

Now, to do this, I use Saxon9HE, which is the opens source version of Saxonica’s (Michael Kay’s) XSLT processor. It’s easy to use, very fast, supports the latest versions of everything, and free.

I’ll follow up this post with another soon about how to do just that. I wanted to post this step first so as to not overwhelm someone who is learning (nor give me an excuse to put off posting anything).

Most notably, to me, is regular expressions. However, the RxFind utility is a great way to add regular expression search and replace functionality to your Windows batch files and I use it a lot.↵

Covering Stairway to Heaven

Posted on Sunday, January 27, 2013 by Jason Coleman

Long before Wayne’s World, it was unwritten rule that no one should ever cover “Stairway to Heaven” by Led Zepplin. Everyone’s heard it too many times and no one can do it justice, anyway, so what’s the point?

Well, if you are going to break that rule and you’re going to do it front of the three remaining band members, the President of the United States, the First Lady, and all rest of the audience at the Kennedy Center —who happens to be honoring Led Zepplin late last year— then you’d best perform just like Ann and Nancy Wilson (of Heart), Jason Bonham (son of the late Led Zepplin drummer, John), and the rest of this amazing band.

Wow. Check out Yo-Yo Ma grooving out to the guitar, the members of Led Zepplin completely entranced, and Robert Palmer shedding a tear after the end!

Okay, now let’s all go back to never covering this song. We’ll revisit it again in another 1/3 century.

The Hobbit: An Unexpected Journey

Posted on Monday, December 24, 2012 by Jason Coleman

During trips to visit grandparents for the Christmas holiday when I was a kids, my family would typically go out to see a movie at least one night. It gave us all something to do and I’m sure gave the adults a much-needed break from watching us kids. I was finally able to take Angela out to see a movie around Christmas this year; something I’m pretty certain she never did as a kid.

We went to see The Hobbit: An Unexpected Journey¹, which of course is the first part of trilogy of films dedicated to the J.R.R. Tolkiens first novel.

So, as not to bury the lead, we both liked it. I can’t say that either of us loved the film and it didn’t have the great impact that the first film in the Lord of the Rings trilogy had for me. But, it was a pretty solid adventure film and it’s hard for me to not have a fond feeling for anything based in middle earth.

The Downside

My biggest dislike of the film was generally just the overall tone. The Hobbit, or There and Back Again was a children’s book and while it certainly had all the fantasy trappings of LotR (and more: dragon!), it was notably more light-hearted. I don’t recall a sense of real danger from Golem; at least no more so than the trolls trying to eat the dwarves.

Of course, there’s simply no way that this movie could have been a child-friendly, light-hearted adventure movie. By making LotR first the tone of the world had been established and you can’t lighten it up easy. Else you get Ewoks or, God forbid, Jar-Jar Binks. And no one wants that from the dwarves of Middle Earth.

The Upside

So, in order to make a movie based on The Hobbit as epic in nature as the LotR, Peter Jackson and company had to do what works for so many movie franchises: a prequel!

Okay, so The Hobbit came before the LotR, so of course it’s a prequel, right? Well, not exactly in the sense we think of movie prequels. Hobbit was written before LotR and with a very different audience and purpose from the author. LotR was an epic myth which keyed off of a single incident in the Hobbit (the ring), which was much more of a simple fantasy adventure tale. To put it another way, if you were a school child in Middle Earth, studying the story of the Hobbit would be akin to learning about some European land feud resulting in a minor war. The events of LotR would be world-changing, like studying the and fall of the Roman empire.

Further, The Hobbit just wasn’t that long of a story. At just under 100,000 words, it’s well in the novel territory, but it’s less than a quarter of the entire LotR trilogy of books (which were really intended as a single novel, anyway). It surely could have been one heck of a single movie, and no one was that surprised when it was announced as two movies. However, three movies really seemed like a stretch. And honestly, the first film could have easily trimmed off a bit in terms of overdone action sequences.

Fortunately, it seems that much of the plot padding in the Hobbit trilogy is coming from canon sources like the appendix of The Return of the King and perhaps The Silmarillion. So there’s probably not a lot of content created here just for the film, though there is no shortage of the extended, elaborate action sequences (most of which involve orcs or goblins) that Peter Jackson loves to embellish on. However, his admiration (and that of his partners in writing & producing) for the source material is evident.

Given just how dry much of Tokein’s other writing was — there’s a damn good reason The Silmarillion or the later posthumeous works of Unfinished Tales and History of Middle Earth never became popular — some added character depth and lively adventuring from Peter Jackson et al. might be just the thing to bring them further to life. Who knows, there may well be a third trilogy in there somewhere. Of course, none are ever going to be as epic and amazing as the Lord of the Rings, including The Hobbit.

In good ‘ole 2D. I’m not only a cheapskate date, I also don’t care for 3D unless it has something to add, which I didn’t think it would for this film.↵

Thomas Carroll Coleman, Sr. 1918-2012

Posted on Friday, November 23, 2012 by Jason Coleman

My grandfather, Carroll Coleman, died this afternoon in hospice in Naples, FL. With is wife, Joan, and two of his children, Kathleen and James him, he passed peacefully after a short decline with pneumonia.

His 94 years were a life spent overcoming much to achieve the American dream. Growing up during the Great Depression in Arkansas, he served in the US Army as an x-ray technician and worked partly on farms until he eventually found work in travelling sales. He married his sweetheart, Ruth (my grandmother), and made a career in sales, purchasing, and management with regional department stores. After losing my grandmother in June of 2001, he later relocated to Southwest Florida where he met Joan, with whom he spent many happy years.

There will surely be a more complete and accurate list of facts of his life to come. But what that brief description fails to capture was the remarkable personality and intelligence of a man who was the definition of a family patriarch. A man who treasured his family and loved to experience life. His optimism and wisdom were an inspiration. To me, he was the epitome of style, class, and intelligence. I looked up to him and any time we were together I saw everyone else doing just the same.

He had a rich life and was relatively well for almost the total of his years. He left peacefully and with loved ones nearby. I choose to think he is reunited with Ruth and at rest.

I am truly grateful for this remarkable man and all that he did for his family.

Batch File Output in MadCap Flare

Posted on Tuesday, November 20, 2012 by Jason Coleman

I have a couple of products which I document using MadCap Flare to generate about two dozen help files and another half-dozen PDFs. These outputs are spread across multiple Flare projects which I inherited. Producing a full set of output for a release can prove to be nearly a full day’s worth of effort so I finally got around to creating a single Windows batch file to take use of the command line interface for Flare. Flare has had the command line feature for a few years now, but regrettably, I just never took the time to learn it. It’s actually very simple to implement, even if you’re not that familiar with writing batch files or the idea of the command line scares you off a bit.

Tools Used

First, I should point out that to further streamline my work, I’ve implemented a couple of other tools besides just Flare. These are all free, open-source tools which I highly recommend you having in your tech-writer toolkit¹.

7-Zip – The best compression utility out there. The command line interface is easy to wrap a lot of files into a compressed archive (variety of formats, including .zip).
NcFTP – A very easy-to-use FTP which has some command line utilities capable of transfer in passive mode (required for our FTP behind a firewall).
Notepad++ – A great text editor which has syntax highlighting for batch files.

And of course Flare. However, you could also easily integrate much of the same workflow into using the DITA Open Toolkit as well as any other help authoring tool with a command line interface.

Set Up

I prefer to use dates in my archive file names just to make things clear for the teams downloading them what ‘version’ it is. Sure, we could just check timestamps, but this just makes it more obvious. I use the international data format — YYYY-MM-DD — as the prefix for my titles and I wanted this automated into my batch file. However, as my region is US on my Windows machine, I need to just change the short date format in the Control Panel to this format. That way, I can use the %date% environment variable to always input the current date when the archive is created.

Aside from that, installing the above tools is all that is required.

Creating the Batch File

Notepad++ can be used to create and edit the Batch file. Simply create a new document and save it (somewhere convenient) with the .bat file extension. This also indicates the file type to Notepad++ so the syntax is highlighted appropriately (simply makes editing easier).

I want to place my outputs in a Zip archive for the convenience of labeling them all with the current date and placing onto a FTP server for other teams to download. So I set a variable to include the current date:

set ZipOut=C:\Documentation\Output\%date%_my_files.zip echo %ZipOut%

(The second line just outputs the same back to me so I can verify the date string was as intended)

Next, I change the directory to the MadCap Flare installation:

cd\Program Files (x86)\MadCap Software\MadCap Flare V8\Flare.app

Then I can use the command line entry — madbuild — to initiate builds of any number of Flare projects and targets (which are individual outputs from a single-source Flare project).

madbuild -project "C:\Documentation\Product\ProductHelp_A\Product_A.flprj" -log true -target "Product_A HTML Help" madbuild -project "C:\Documentation\Product\ProductHelp_B\Product_B.flprj" -log true -target "Product_B HTML Help" madbuild -project "C:\Documentation\Product\ProductHelp_C\Product_C.flprj" -log true -target "Product_C HTML Help"

Next, I want these three compiled HTML Help files to get placed into the ZIP file I named in my variable. This uses the command line interface for 7-Zip:

cd\Program Files\7-Zip 7z a -tzip %ZipOut% @C:\Documentation\Output\Product_file_list.txt

Where Product_file_list.txt is just a plain text file containing the absolute file path and file name of each of the compiled HTML Help files. It’s described in detail in the 7-Zip help, but essentially the entire file path for each file to be included is on a line in the text file. No special syntax or separators required.

Lastly, I want to transfer the ZIP file over FTP to a convenient place for the rest of the team. The default Windows FTP program cannot run in passive mode, which is required to navigate a firewall. However, the Linux FTP client NcFTP has been ported to Windows and has a command line interface which is more flexible.

cd\Windows ncftpput -F -u username -p password ftp.mydomain.com /Product/ %ZipOut%

Running the Batch File

Just save the file in your text editor. All that is needed to run it is to simply double-click the .bat file in Windows Explorer. The command line window will open, execute each line in order, and close upon completion.

It would be easy to also use Windows to schedule running the same thing nightly or weekly if you need to regularly post updates of your work.

There are OS X and Linux equivalents to these, but not to Flare, which is why I’ve limited this to Windows.↵

Health Insurance

Posted on Monday, November 19, 2012 by Jason Coleman

I hear and read a lot of complaints about health insurance rates from friends and family. I figured since I had some good news, it was only fair to post it. Our rate are going up in 2013, by around 3.6% by my calculation. Out of our pocket, this results in $10 more per month. Not great, but given some of the leaps in insurance rates in past years, that’s pretty low.

But, next year my employer’s plan will not require a copay for preventative care visits. Or, in other words, those are 100% covered¹. As my entire family is covered for medical & dental, that’s four physicals and eight dental cleanings (we go every six months). In other words, while our annual premium portion is going up $120, we’ll be paying at least $180 less than previous years.

So, thanks to my employer, my insurance company (we have Aetna), and to President Obama for making my insurance costs go down for this first time ever!

I’m fairly certain this is in order to comply with Obamacare, but this if they want to claim this as a new benefit, it doesn’t really matter to me.↵

SS Coleman

Posted on Monday, November 5, 2012 by Jason Coleman

My father retired from his law practice about a year ago. More recently, he and his wife sold their home and moved west to Arkansas. In this process, he’s been trying to both downsize their household as well as get rid of many years worth of office items. My brothers went to help clean out a storage unit a couple of months ago and returned with one of the ’50s-era, metal office desks that my father had in his law office.

The desk is battleship gray, with quite a few dings and scratches. However, it’s very sturdy (as it’s made of approximately 1,000 tons of sheet metal) and still in pretty good shape. The damage it has is more along Wabi-sabi¹ than disrepair, so I’m okay with it for the most part.

This, along with a similar style desk, were in my father’s office since I was very young. Also, since I was very young, I’ve always been fascinated with not only space exploration and technology, but the aesthetic that is associated with those things. If you can imagine the desk that an engineer at either NASA or IBM might have sat at sometime in the early ’60s, you’re thinking of a desk like this.

This particular desk has an interesting feature where a corner of the desk is lower than the work surface to accomodate a typewriter (no doubt, sized for a 1961 IBM Selectric).

This desk also has drawers (!), unlike my old wood desk. I just need to clean up the glides a bit. I’ve of course never heard a dying pterodactyl, but I think I have a very good idea what one might sound like based on the bottom drawer opening.

It’s worth noting here that my wife doesn’t particularly care for the desk. Probably for two reasons: 1) She (rightly) notes that it really doesn’t fit in with pretty much any other furniture in my office, let alone the rest of the house and 2) she —at some fundamental level— doesn’t recognize the concept of Wabi-sabi. That is, not that she doesn’t get the idea, just that in her opinion, it’s just wear and tear that should be fixed rather than aesthetic appeal.↵

Elections

Posted on Friday, November 2, 2012 by Jason Coleman

It doesn’t amount to making any difference, but as a rule I never vote for a candidate who is running unopposed. Sadly, this happens a lot. Many of the races for Congress I’ve ever voted in were this way.

There was one election in Richmond, Virginia in which I literally voted for no one (though I think I did answer y/n on one of the question votes). Really doesn’t do much to motivate one to vote at all when it really doesn’t matter, right?

So I’m very glad to see some competition —no matter how anemic— on my ballet this November. I’m no fan of Rep. Blackburn or Sen. Corker. Though some of their competition are no-chance, fringe candidates; folks like that have occasionally won races.

So if you haven’t already voted, go vote this Tuesday. Even if you have to skip over some one-horse races, it always matters.

Five Fun Things Friday – Father’s Day 2012 Edition

Posted on Friday, June 22, 2012 by Jason Coleman

Oh, it’s been a while, hasn’t it?

My last post was mid-March. Since then, I’ve been the busiest I’ve been my entire working career. Problem #1 is that I have a hard time saying no to colleagues. Problem #2 is —one of the main causes of #1— is that I really enjoy all of the projects I’m involved with.

So, after 2-1/2 months of 60+ hour weeks, I’ve scaled back my work-load (thanks very much to my supportive boss & colleagues along with my very talented summer interns). As a result, I’ve had a little time enjoy some cool things in the past week or so:

Father’s Day – This year was terrific; much more than I deserved really. Angela made me a delicious breakfast of chocolate chip pancakes. I got drill press (see the following) from Angela and some swim goggles from the kids. And I got to spend almost the entire day with the family (except when the kids were out shopping for swim goggles & I was playing with the drill press).
Woodworking – As the kids have gotten bigger, they’ve started to out-grow their play & crafts table. So Angela asked me to make it tall enough to store toys underneath & allow them to sit at the table for crafts. So, using my new drill press, I hollowed out some glued-up MDF blocks to create leg extensions I could lag-bolt into the table legs. It’s a bit rough, but was the first thing I’ve got to make in quite a long time.
Hunger Games – I completed Mockingjay, the final book of Suzanne Collins’ excellent post-apocalyptic YA novels. I haven’t seen the film based on the first book yet, but I can certainly recommend the books. They are lightening fast to read, thanks to Collins’ choice of first person narrative and her intense action.
Prometheus – Okay, this was technically last week, but I went with my brother, Stephen, to watch Ridley Scott’s return to science fiction. There has been some intense discussions about this movie. I, for one, really enjoyed it. I’ve been a big fan of the Alien series for a long time, including the comics. I can’t quite say it was a perfect film, but it met my every expectation and it certainly one of the year’s best sci-fi films.
Swimming – So, our kids have been taking swimming lessons for the past few summers and Angela recently competed in her first-ever triathlon. As a result, we’ve found ourselves at the pool quite a bit. As our kids are old enough to actually swim around some and Angela has been also learning how to properly swim for a race, I’ve actually gotten to swim some, as well. Admittedly, I’m surely the worst swimmer in the family. However, it’s a lot of fun to get to spend time in the pool with the family and a great way to beat the crazy heat here in TN.

A Decade of Us

Posted on Sunday, March 18, 2012 by Jason Coleman

This post is a technically a couple of days late for our tenth wedding anniversary, but you can skip all the stereotypes about the husband who forgets that sort of thing. That’s not at all why it’s late. Rather, Angela and I were off on a family vacation and not bothering to post it to the internet (that occasionally happens to me).

Despite the most wonderful ten years of my life (and hopefully Angela agrees on that), we didn’t celebrate with a fancy dinner. Instead—in the wee early hours of the morning—we found ourselves sharing take-out pizza in a hotel room bed with our kids while clinking cheap-yet-overpriced wine in a pair of plastic cups. We decided to take the kids on our first big family vacation and went to Disney World in Florida. While not a traditional date, even for us, we’ve always done things our own way and therefore it seemed fitting. Besides, our kids are as much a part of our marriage as anything so it felt right to include them (though they didn’t get any of the wine; there wasn’t enough in the little hotel bottle to share). Our actual anniversary was spent on a long drive home. We joked we’d spend an hour on the road for each year we’d been married. In a way, also fitting as we spent the day after our wedding on a long car ride back home to Virginia. I don’t think we’ll try that for any future anniversaries, though¹.

Not to stretch the driving metaphor too far, but it has been an amazing journey these past ten years (and the years together even before then…). I’ve got a friend and partner. Anytime I described a dream of mine, she’s never tried to stop me but instead asked “when do you want to get started?” She’s not only supported me but pushed me to always be better and do more. Everything good that has happened to me since we’ve been together has been in a very large part because of her. To have had a friend like that even for a short time is a gift and she’s been married to me for ten short years (and two days).

Here’s to ten times ten more, should we be here for so long.