2009-09-11

Adding Zipping and unzipping capabilities to Java.io.File via Groovy MOP

this post has an update : Playing again with zip and Groovy MOP ( and spread operator)


[First technical post, and I don't know how to begin... Please be kind with me]
So, as I'm writing a Plugin System in Groovy, I needed to zip and unzip so-called 'Bundles' which contain config files, Plugin jars and dependencies.

I didn't want to make a classical FileUtils or BundleUtils class, and I decided to try Groovy MOP (Meta Object Protocol) to directly modify the java.io.File so I could write
new File('/path/to.file').zip(/path/to.zip)

First thing to do is adding capability when you need to. As an example, I decided to load the capability statically in a Class. You could either have a bootsrap class that loads your metaprogramming functionnalities before starting your App. This is your choice
static{
  bootstrapPluginSystem()
}

private static void bootstrapPluginSystem(){
   //zipping methods here
   ..................
   //unzipping methods here
   ..................
}

Zipping files is easy in Java and is performed via the ZipOutputStream class.

One of the grooviest groovy goodies is the ability to work seamlessly with streams without boilerplate resource closing code
Simply use the withStream method, and Groovy manage the resource for you. How great it is !!!

Groovy MOP is wonderful to add methods to Class at Runtime. The simple example I take here demonstrates the power of this technique

Zipping is quite easy :
//define zip closure first, then allocate it
 //it is needed to make the recursion work
 def zip
 zip = { ZipOutputStream zipOutStream,File f , String path->                        
    def name = (path.equals(""))?f.name:path + File.separator + f.name
    if(!f.isDirectory() ){                            
      def entry = new ZipEntry(name)
      zipOutStream.putNextEntry(entry)
      new FileInputStream(f).withStream { inStream ->
        def buffer = new byte[1024]
        def count
        while((count = inStream.read(buffer, 0, 1024)) != -1) {
          zipOutStream.write(buffer,0,count)
        }
      }
      zipOutStream.closeEntry()
    }
    else {
      //write the directory first, in order to allow empty directories
      def entry = new ZipEntry(name + File.separator)
      zipOutStream.putNextEntry(entry)
      zipOutStream.closeEntry()                            
      f.eachFile{
        //recurse
        zip(zipOutStream,it,name)
      }
    }
 }

 File.metaClass.zip = { String destination ->
    //cache the delegate (the File Object) as it will be modified 
    //in the withStream closure
    def input = delegate            
    def result = new ZipOutputStream(new FileOutputStream(destination))
    result.withStream {zipOutStream->   
      //recursively zip files             
      zip(zipOutStream,input,"")
    }
 }


Code is almost self explaining...
Unzipping comes with less efforts :
File.metaClass.unzip = { String dest ->
 //in metaclass added methods, 'delegate' is the object on which 
 //the method is called. Here it's the file to unzip
 def result = new ZipInputStream(new FileInputStream(delegate))
 def destFile = new File(dest)
 if(!destFile.exists()){
   destFile.mkdir();
 }
 result.withStream{
   def entry
   while(entry = result.nextEntry){
     if (!entry.isDirectory()){
       new File(dest + File.separator + entry.name).parentFile?.mkdirs()
       def output = new FileOutputStream(dest + File.separator 
                                         + entry.name)                        
       output.withStream{
         int len = 0;
         byte[] buffer = new byte[4096]
         while ((len = result.read(buffer)) > 0){
           output.write(buffer, 0, len);
         }
       }
    }
    else {
      new File(dest + File.separator + entry.name).mkdir()
    }
   }
 }
}


All that is not really thoroughly tested, but I managed to zip and unzip the grails project directory without problems. I can also unzip with Ark files zipped by these lines of code

Here are minimal tests :
import groovy.util.GroovyTestCase
/**
 *
 * @author grooveek
 */
class ZipTest extends GroovyTestCase{
 // load the static block to add zip capabilities
 static def bootstrap = new BootStrap()

 void testZipUnzip(){
   def fi = new File('testfiles')
   if (!fi.exists()){
     fi.mkdir()
   }
   def writer = new File('testfiles/ziptest.txt').newWriter()
   100.times{
     writer.writeLine("$it")
   }
   writer.close()
   //testing zipping/unzipping
   def fileToZip = new File('testfiles/ziptest.txt')
   fileToZip.zip('testfiles/toto.zip')
   def zipToUnzip = new File('testfiles/toto.zip')
   zipToUnzip.unzip('testfiles/ziptestdir')
 }

 void testZipUnzipWithDirectories(){
   def fi = new File('testfiles')
   if (!fi.exists()){
     fi.mkdir()
   }
   new File('testfiles/toto/titi').mkdirs()
   def writer = new File('testfiles/toto/titi/ziptest.txt').newWriter()
   100.times{
     writer.writeLine("$it")
   }
   writer.close()
   //testing zipping/unzipping
   def fileToZip = new File('testfiles/toto')
   fileToZip.zip('testfiles/toto2.zip')
   def zipToUnzip = new File('testfiles/toto2.zip')
   zipToUnzip.unzip('testfiles/ziptestdir2')
 }

 void testZipUnzipWithDirectoriesAndMultipleFiles(){
   def fi = new File('testfiles')
   if (!fi.exists()){
     fi.mkdir()
   }
   new File('testfiles/tata/titi').mkdirs()
   5.times{
     def writer = new File("testfiles/tata/titi/ziptest${it}.txt").newWriter()
     100.times{
       writer.writeLine("$it")
     }
     writer.close()
   }
   new File('testfiles/tata/titi/toto').mkdirs()
   5.times{
     def writer = new File("testfiles/tata/titi/toto/ziptest${it}.txt").newWriter()
     100.times{
       writer.writeLine("$it")
     }
     writer.close()
   }
   new File('testfiles/tata/titi/toto/tutu').mkdirs()
   //testing zipping/unzipping
   def fileToZip = new File('testfiles/tata')
   fileToZip.zip('testfiles/toto3.zip')
   def zipToUnzip = new File('testfiles/toto3.zip')
   zipToUnzip.unzip('testfiles/ziptestdir3')
}
        

 void testZipBigAndDeepDirectory(){
   def fileToZip = new File("/home/grooveek/grails-1.1.1")
   fileToZip.zip("testfiles/grails.zip")
   def zipToUnzip = new File('testfiles/grails.zip')
   zipToUnzip.unzip('testfiles')
 }
}

I hope you enjoyed reading
Don't forget to let a kind word (or something not so kind if that's your mind)you want

See you soon

@grooveek

8 comments:

  1. Impressive !, Groovy really rocks, I have a colleague who I''m sure can use this (zipping log files), I will send him the link, I think you should also post about this on gr8forums.org on the section "tips and snippets" !

    ReplyDelete
  2. I'm not really understanding your comment... OSGI has nothing to do with zip, huh ?
    You may think about my module system ? It's a bit complicated to explain a design decision in a few words, but we develop a distributed module system with peer to peer loading of modules on large clusters. OSGI is not designed to handle such use case, and we ran into troubles while evaluating it
    I hope I answered your question
    grooveek

    ReplyDelete
  3. The first part of starting any new site is picking the niche you want it to be in hotel marrakech. My biggest criteria for this case study was finding a niche that should be fairly easy to get some traction in rapidleech servers, so I went for something pretty obscure pnr status. I don’t know how much money is here, so I’m taking a chance therescrapebox. But all techniques stay the same adwords coupon.

    ReplyDelete
  4. Thank you for posting this tutorial.

    ReplyDelete
  5. some of the tests in ZipTest did not work for me. I had to change the beginning of the 'else' block in the zip closure like so:

    //write the directory first, in order to allow empty directories
    def entry = new ZipEntry(name + '/')

    ReplyDelete
  6. Doesn't work so well with non UTF-8 encoding....

    ReplyDelete

Share your feelings about this post, or this blog