Run a List of Gradle Tasks In Specific Order

I recently set out to add a few “synonym tasks” to our gradle based build. This is something which is quite easy to do in the ant world, and it seemed like a concept that would be very easy to translate into gradle.

What’s a synonym task? Its a task that just runs a series of other tasks, in a specified order (it could also be called an “alias”). For example, instead of typing this long series of tasks:


gradlew clean assemble runProvisioner stopTomcat installTomcat deployToTomcat startTomcat

Its much more convenient to just type something like this (where “all” is an alias for the above list of tasks) :

gradlew all 

Now – if you explicitly tell Gradle which tasks to run via the command line (by typing them out each time), it will respect the ordering you provide. It will execute each of those tasks (along with each of their dependencies) in the order in which you specified via the command line.

Given that – you’d think it would be pretty simple to create an alias task which just calls those things in the expected order.

Not so.

This turned out to be a lot harder than it should have been (which is what drove me to write this, on the chance it might help someone else).

Why is defining a synonym task hard?

Gradle task execution order is typically dependency driven. Left to its own devices, Gradle likes to determine the order in which to run tasks based on a DAG (“Directed Acyclic Graph).

In Gradle, dependencies aren’t necessarily ordered.

Its seems that this has been a point of contention for a while. Many of the discussions I found reference GRADLE-427 (which was opened in 2009).

In recent versions of Gradle, they added a .mustRunAfter method which allows a task to say, “When this other task is going to be run anyway, I should run after it.” That was meant to address the 427 issue, but doesn’t really seem like a great solution for the synonym / alias use case. Here, another Gradle user laments the same problem, and one of the Gradle devs chimes in with a utility function that can provide the equivalent of ordered dependencies:

http://forums.gradle.org/gradle/topics/mustrunafter_not_a_usable_fix_fot_the_dependson_ordering_bug

My first solution to the build aliases problem was to solving it similarly to the way the above post suggests. I wasn’t happy with this solution (its a bit too heavy), but I’ll share it here, just for completeness :

void wireOrderedDependencies( task, others) {
    task.dependsOn(others)
    for (int i = 0; i < others.size() - 1; i++) {
       others[i+1].mustRunAfter(others[i])
    }
}

task all(description: 'an alias for :  clean assemble stopTomcat runProvisioner installTomcat deployToTomcat startTomcat') {
}

//--- set up the "all" target's dependencies IN ORDER in this List
def all_dependenciesInOrder = []

subprojects.each () { sp ->
   all_dependenciesInOrder.add sp.clean
}
all_dependenciesInOrder += [project(':myapp-provisioner').assemble, project(':myapp-web').assemble, stopTomcat, runProvisioner, installTomcat, deployToTomcat, startTomcat]

wireOrderedDependencies(all, all_dependenciesInOrder)

This initial solution seems like something that wouldn’t scale very well. For example, things would get very messy if we had 8-9 different aliases.

As I researched the issue a bit further, I found what seemed like a much cleaner suggestion referenced within one of the (many) comments on GRADLE-427. It was this suggestion.

That seemed like a much cleaner approach, and is the basis for what I ended up doing. Here’s the cleaner solution :

//--- build aliases : define a synonym here if you want a shortcut to run multiple targets

def buildAliases = [
   'all' : ['clean', 'assemble', 'runProvisioner', 'stopTomcat', 'installTomcat', 'deployToTomcat', 'startTomcat'],
   'rebuild' : ['clean', 'assemble']
]
def expandedTaskList = []

gradle.startParameter.taskNames.each {
   expandedTaskList << (buildAliases[it] ? buildAliases[it] : it)
}

gradle.startParameter.taskNames = expandedTaskList.flatten()

println "\n\n\texpanded task list: ${gradle.startParameter.taskNames }\n\n"

This solution works off the fact we discussed earlier: If you explicitly invoke Gradle from the command line with a specific list of tasks, it respects the supplied ordering for those tasks. gradle.startParameter.taskNames contains the list of tasks that the user supplied on the command line.

This solution simply iterates the user supplied list of task names, and if it finds one of these task names in the buildAliases map, it “expands” the task name to be the associated list of tasks. Much better.

A Very Useful Tool for Cognos Admins

MotioPI is a great (free) tool for Cognos Administrators.   If you are responsible for one or more Cognos environments, I highly recommend it.     In short, its designed to provide Cognos administrators an easy way to gain insight into the content and configuration in their content store.   

For example – you can quickly Find all Schedules associated with a given area of your content store,  Batch Validate a series of Cognos Reports, or quickly snapshot your various Cognos administrative settings (dispatchers, report services, etc.).

Note – by and large, the free version of MotioPI is read-only, the one exception being the Orphaned Objects panel – which lets you re-assign Cognos  content (e.g. My Folders data) to another user.   

Motio also has a professional version which lets you do mass updates to Cognos content / configuration (e.g. search and replace across many reports, bulk management of user preferences / group memberships, replicate properties across many Cognos objects, etc.).

Auto Create Quartz Tables at Startup

I recently needed to add Quartz support to a Spring project, along with a mechanism for automatically creating the Quartz tables at application startup. I did a bit of googling on how to get Quartz to auto-create its tables, but didn’t find anything. Here’s what I ended up doing to solve the problem.

First, lets consider the basic configuration of a Quartz Scheduler in Spring. If you’re planning on storing Jobs in a relational database, then your entry will look something like this :

   <bean id="quartzScheduler" class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
      <property name="dataSource">
         <ref bean="dataSource"/>
      </property>
      <property name="transactionManager">
         <ref bean="transactionManager"/>
      </property>
      <property name="quartzProperties">
         <props>
            <prop key="org.quartz.jobStore.tablePrefix">QRTZ_</prop>
            <prop key="org.quartz.jobStore.selectWithLockSQL">SELECT * FROM {0}LOCKS UPDLOCK WHERE LOCK_NAME = ?</prop>
         </props>
      </property>
   </bean>

At this point, you could start up the application, but you’d get JDBC errors when Quartz tries to access its tables. You could pop out to a SQL console and manually create these tables, but you’d be faced with the same problem each time you deployed to a new environment.

To remedy this, we’ll make use of Spring’s DataSourceInitializer class, which will run designated SQL scripts when deployed :

<!--
   This will execute SQL scripts to recreate the quartz tables at 
   appserver boot time. 
-->
<bean id="quartzDbInitializer" class="org.springframework.jdbc.datasource.init.DataSourceInitializer">
  <property name="dataSource" ref="dataSource"/>
  <property name="enabled" value="true"/>
  <property name="databasePopulator">
    <bean class="org.springframework.jdbc.datasource.init.ResourceDatabasePopulator">
      <property name="continueOnError" value="true"/>
      <property name="ignoreFailedDrops" value="true"/>
      <property name="sqlScriptEncoding" value="UTF-8"/>
      <property name="scripts">
        <array>
          <value type="org.springframework.core.io.Resource">
            classpath:META-INF/quartz/oracle/drop-quartz-tables.sql
          </value>
          <value type="org.springframework.core.io.Resource">
            classpath:META-INF/quartz/oracle/create-quartz-tables.sql
          </value>
        </array>
      </property>
    </bean>
  </property>
</bean>

The above will execute two sql scripts each time the application is started :

  • META-INF/quartz/oracle/drop-quartz-tables.sql
  • META-INF/quartz/oracle/create-quartz-tables.sql

Note – you’ll probably want to remove / comment out the drop-quartz-tables.sql eventually (it will drop your quartz tables each time you restart the application – ok for kicking off development, but not viable long term).

Just one more small thing left to do : we need to add a dependency between our quartzScheduler bean and the quartzDbInitializer bean (so that the quartzDbInitializer bean will be instantiated BEFORE the quartzScheduler bean). To do this, simply add a “depends-on” attribute to the quartzScheduler bean (as shown below) :

   <bean id="quartzScheduler" class="org.springframework.scheduling.quartz.SchedulerFactoryBean" depends-on="quartzDbInitializer">
      <property name="dataSource">
         <ref bean="dataSource"/>
      </property>
      <property name="transactionManager">
         <ref bean="transactionManager"/>
      </property>
      <property name="quartzProperties">
         <props>
            <prop key="org.quartz.jobStore.tablePrefix">QRTZ_</prop>
            <prop key="org.quartz.jobStore.selectWithLockSQL">SELECT * FROM {0}LOCKS UPDLOCK WHERE LOCK_NAME = ?</prop>
         </props>
      </property>
   </bean>

That’s it.

FlipShare MinoHD “Camcorder Full” and “Camcorder Empty” Problem

We purchased one of those FlipShare MinoHD camcorders about a year ago, and I must say its a really nice little device. Very affordable, very small, easy to use, and makes great videos.

My wife was cleaning off some old videos the other day and somehow managed to get the device into an odd state in which its totally unusable. Here are the symptoms:

  • If you press the record button, you get a message saying “Camcorder full”.
  • If you press the trash button, you get a message saying “Camcorder empty”

So basically – you can’t record anything (because the MinoHD thinks its full), and you can’t empty the trash (because the MinoHD thinks its already empty). Awesome!

After a few minutes of debugging, I came to the following conclusion. When you plugin the MinoHD to your computer, it basically acts like a little hard drive (formatted as FAT32). It appears, that when in this unusable state, all of the videos that you thought you deleted are actually still on the drive, in its recycle bin.

On an XP Professional laptop with hidden files / folders turned on, you’ll see something like this (note all the large MP4 files still hanging out under the .Trash folder) :

FlipVideo Camcorder Full / Camcorder Empty problem

I simply deleted all the MP4 files that were still in this trash folder, then ejected the flipshare, and things went back to working fine for me.

Note – I was expecting to find an “empty recycle bin” type context option when right clicking on the drive or the .Trash folder, but didn’t see one.

To turn on hidden files / folders in windows explorer, you’d go to Tools | Folder Options, then press the View Tab. Make sure the “Show hidden files and folders” radio button is selected (this is one of the first thing I do anytime I’m dealing with a new Windows install).

Windows Explorer - Show Hidden Files and Folders

Confused by Cognos 8.4.1 Install Files…? This page can help.

It seems like in the last 9 months, the IBM acquisition of Cognos has really changed a lot of the way things are done for partners and customers. For example, the manner in which you get support and downloads has changed dramatically.

Maybe its just me, but finding the downloads for the Cognos product you want from the IBM support site seems much harder than it used to be, and when you do finally locate it, you now use “download director” to pull down all of the associated “assemblies”.

With Cognos 8.4.1, it seems like the actual installables have also changed quite a bit. Yesterday I pulled down 8.4.1 from the IBM support site, and once download director was done doing its thing, I was a bit taken aback by the number and names of files it pulled down (screenshot below).

At first glance, I’m guessing most will have no idea which of these files corresponds to the component, platform and version of the Cognos product they’re after. For example – its not immediately apparent that “CZA86ML.tar.gz” actually contains the install for “IBM Cognos 8 Business Intelligence Server 32-bit 8.4.1 Windows Multilingual”.

Eventually I found this page on IBM’s site which helped clear up the confusion (partial screenshot below).

I think Google is going to take a serious bite out of Apple…

Part of me wonders if we’ve seen a “local maximum” on Apple.

I think their escalating rivalry with Google is going to really hurt them in the long run. Here are some of the key fronts in my opinion.

The Android Platform : Google’s Android mobile operating system is the open source platform behind many smart phone competitors to the iPhone. It’s also starting to show up in other non-phone devices like netbooks and the Barnes and Nobles Nook (E-Reader).

Android is an open platform, being used on a wide variety of devices, with backing from a diverse ecosystem of contributors. Apple’s iPhone is a closed and tightly controlled platform being developed by one company (some might throw out a Linux vs. Windows analogy).

One perspective which I find a bit ironic : By giving smart phone providers a rich, stable core to build upon, Android should free them up to focus more on consumer oriented features (rather than expending huge amounts of effort on OS level plumbing). From the smart phone provider perspective, isn’t this kind of similar to how Apple leveraged a solid BSD Unix based core to build the very consumer friendly OS X…?

It will be interesting to watch the evolution of these two platforms in the coming years. One might speculate that the diverse ecosystem leveraging Android may drive more innovation.

Google’s Online Music Service : There’s widespread speculation that Google is launching an Online Music Service which will compete with iTunes.

Maybe its just me, but it seems like consumers have a lot of pent up animosity at Apple over iTunes. iTunes has pissed off users for years (try googling for I hate iTunes and marvel at the number of results).

Personally – iTunes drives me nuts, but I’m stuck with it (I have an iPhone). Over the years, it has lost my music, made it difficult for me to share music I purchased across multiple devices, refused to read metadata on music I imported via non-iTunes software, etc.

iTunes is at version 9. Its been around for years and has a huge user base. There’s no reason why Apple shouldn’t have fixed many (if not all) of these issues. Certainly there’s been lots of feedback – maybe its fallen on deaf ears.

Apple will no doubt continue to be the dominant player in online music sales, but it can’t be good that a company with Google’s clout and reputation is getting ready to compete with them in this space. I think their window for easy money in digital media distribution is shrinking.

Public Opinion : Maybe its just me, but it seems tide of public opinion regarding Apple is starting to turn a bit. For years they have been the cool, innovative underdog. At times they now seem more like a condescending incumbent.

For example – consider their draconian rejection policies for third party applications submitted to the iTunes App Store. As a developer, I can tell you I would be absolutely done with Apple if I worked my tail off on an iPhone App for months only to have them reject it with basically a “because we said so” response. This will most certainly drive innovative app developers away from Apple (and to a more open model).

From a PR perspective, this sort of thing is a certainly a black eye for Apple.

Contrast this with the current public love affair with Google and their “Do No Evil” branding, and I think Google is poised score points vs. Apple in the court of public opinion.

------

Since the above is largely opinion – I thought I’d add a bit about myself : I am not an Apple fanboy or an Apple hater. I own machines with OSX (macbook), XP (netbook) and Ubuntu (primarly laptop) on them. I’m now on my second iPhone (I’ve had a first gen and a 3GS), and I’ve not yet owned an Android based phone. I have yet to develop any iPhone or Android based apps. I don’t own any Google or Apple stock. In short – I have no bones to pick with Apple, these are just observations.

Nice Consolidated HTML Version of All Cognos 8.4 Docs…

Here’s a nice, consolidated HTML version of all of the Cognos 8.4 docs. It seems to span all of the PDFs (Installation, Administration, Authoring, Modeling, etc), has a great, navigable table of contents on the left, and is searchable :

http://publib.boulder.ibm.com/infocenter/c8bi/v8r4m0/index.jsp

I like this far better than hopping between 15 PDF’s.


All Cognos Docs via HTML

Launching Previously Run Webstart Apps Via The Java Control Panel

I’m a huge fan of Java WebStart.

Most people generally launch WebStart based apps by simply hitting a URL which pulls down a JNLP file. A JNLP file is simply an XML descriptor which describes the jars that make up the application, the main class, etc. Once the JNLP file is pulled in, Java takes over and will retrieve the specified jars and then launch the Java application.

To re-launch the app in the future, you can simply bookmark the original JNLP URL in your browser (note to MotioPI users – you can simply bookmark the original URL sent via email and do this with MotioPI).

But suppose you’re in disconnected mode (perhaps waiting in the airport, or on a plane). You can still launch previously executed WebStart based applications by using the Java Control Panel.

If you’re using Windows XP, you can get to this via Start | Control Panel, then double click on the “Java” icon.


Step 1 - Launch Java Control Panel

This will launch the Java Control Panel. From there, select the “View” button (bottom right).


Step 2 - Select the View Button

This will open the Java Cache Viewer. You should have an entry for each WebStart app that you’ve previously run (screenshot below):


Step 3 - Java Cache Viewer

From here, you can right click on any of the cached WebStart apps and do things such as :

  • Run the application
  • Create a shortcut on your desktop to launch the app (so it behaves like a typical client side app)
  • Show the JNLP File
  • Delete the App

An Easy Way to Test against Multiple Versions of IE…

Most web developers have felt the pain of having to test their webapp in N different browsers (and having to work around the quirks in each).

I just spent the better part of the afternoon testing some fluid resize logic in IE7, Chrome and Firefox. I got everything working between those three, but I knew IE6 was probably going to bite me. I only have IE7 installed locally, so I was kind of resigned to testing it later, from a separate machine.

Then a friend pointed me at the following utility which will let you run multiple prior versions of IE, side by side with your existing IE7. Its a breeze to install and works great. You can actually have IE6 and IE7 up at the same time (sweet!).

A 1 minute overview of JSON

JSON is (for the most part) perfectly legal JavaScript syntax, that has been around for quite a while. Think of it as shortcut literal syntax for quickly building a JavaScript object graph.

  • Every Object in JavaScript is an associative array (think Map in Java, a set of key -> value pairs).
  • In JavaScript, the curly braces {} are shorthand for “new Object()”
  • In JavaScript, the square brackets [] are shorthand for “new Array()”

So the following two snippets will build the exact same JS Object graph :

      // In JSON syntax 
      var contact = {
         'name': {
            'first': 'Lance',
            'last': 'Hankins'
         },
         'phone': {
            'mobile': '214.555.5555',
            'work': '972.555.5555'
         },
         'email':  {
            'work':'lhankins@work.com',
            'personal':'lhankins.hankins@home.com'
         }
      };

A non-JSON equivalent :

      // The more traditional NON-JSON way to do the same thing :
      var contact2 = new Object();

      contact2.name = new Object();
      contact2.name.first = 'Lance';
      contact2.name.last = 'Hankins';

      contact2.phone = new Object();
      contact2.phone.mobile = '214.555.5555';
      contact2.phone.work = '972.555.5555';

      contact2.email = new Object();
      contact2.email.work = 'lhankins@work.com';
      contact2.email.personal = 'lance.hankins@home.com';

Since JSON is so good at letting you concisely describe an object graph (hierarchical data), its often a great choice as the output format for server side responses to AJAX calls. On the client side, you can simply “eval” the response text to create the resultant JS object graph.

Here’s another link which also describes JSON in a concise form – and has a funny picture 🙂 Also, this post describes some of the exception cases where JSON is not actually legal JS (JS is a little more restrictive on the allowed characters for property names).