Sunday, January 17, 2010

Ruby and AOP

Right now I am learning Ruby, I focus on Dave Thomas' famous pickaxe Ruby book, last week I just finished the chapter 24: "Metaprogramming".Now I learn the power of Ruby Metaprogramming, right now it is still quite difficult for me, I am sure I will read this chapter over and over in future.

Today I want to discuss my impression of AOP concept in Ruby.
I learned AOP from java, in java you have to use AspectJ or Dynamic Proxy implement AOP. But comapre to Java, I realize Ruby support AOP in the lanuage level:
  • Classes are open: you add new method or attribute for any existing class, or replace the method with new one.
  • Module and Mixin: you can easily add new functionalities by Mixin the module.
  • MetaProgramming: you can dynamically define a method on the fly, using hook method as call back, easily implement before, after and around advice
I feel by using the above technologies, it is quite easy using AOP concept in Ruby, for example you can easily implement a Logging Aspect in a Module, then get it mixin in your application code.That is separating of concern, which is the essence of AOP.
So I will say AOP is built in support in Ruby language level.

Today I found a blog called "Does Ruby need AOP?", the author argue that Ruby metaprogramming is not AOP,because he think AOP is all about semantics, while Ruby metaprogramming is too low level. I feel this argument is too academic, from the pragmatic perspective, I think as long as Ruby can do the job of separation of concern, then it is AOP, then who care about the semantics?

Here are some useful links:
  1. another blog about ruby and APO at 4 line of code
  2. slide: AOP of Ruby

Sunday, January 10, 2010

Customize Your J2ME Emulator

Why do we need to customize J2ME emulator ?
As a J2ME mobile developer, you have to port application to different devices,  while the J2ME default emulator only provide 4 different profiles, which can not meet our needs, and also some platforms like BlackBerry, Nokia S60 etc, the J2ME emulator they provided are really slow to launch, which is really inconvenient. If you can customize the emulator to support the device you are porting, which will improve your developing productivity.

Here I will give you’re my examples: BlackBerry 9000(Bold), LG XENON, Nokia 5800, Samsung Brooklyn. Please check the following table for their specs:
                
Device Name
Key pad
Screen Size
Touch Screen?
Black Berry 9000
QWERTY
480x320
NO
LG XENON
QWERTY
400x240
YES
Nokia 5800
QWERTY
640x320
YES
Samsung Brooklyn
QWERTY
320x240
YES

I will show you how I customize the above 4 devices.

Where are the emulator device profiles located?
 For example, if you installed wireless toolkit 2.5 in c:\WTK25, then the device folder is:
C:\WTK25\wtklib\devices

The mobility package is included in the NetBeans 6.5 or later version, the wireless toolkit folder is like this:
C:\Program Files\NetBeans 6.x\mobility8\Java_ME_platform_SDK_x.x

Create a new emulator device profile
I like to choose Sun Wireless Toolkit 2.5, because its Qwerty Device emulator looks better.   For example, if I want to create a device for Black Berry 9000:
1.  Copy the entire folder of  “QwertyDevice” to a new folder, rename the new folder name to “BlackBerry9000”
2.  Go inside the folder, rename the Qwerty.properies file to BlackBerry9000.properties

Enable the touch screen
The touch screen feature is disabled for J2ME default emulators. If you want to enable it, you need to modify the .properties file. In properties file, find following statement:

      touch_screen=false

Change false to true will enable the touch screen.

Change the screen size
For example, for BlackBerry 9000, its screen size is 480x320, then go to .prperties file, update the following values:

    screenPaintableRegion.width=480
    screenPaintableRegion.height=320

    and ..
    screen.width=480
    screen.height=332

Please note that you need to make sure:
   screen.height >  screenPaintableRegion.height + screenPaintableRegion.y
   In the properties file, screenPaintableRegion.y=11, so the screen.height value is 332.

   If you don’t put the correct value, the emulator will throw exception once it is launched.

Make your customization available in NetBeans IDE
Once You finished the customization, you need to update your NetBeans IDE environment:
  1. Select Tools – Java Platform,  choose Sun wireless toolkit 2.5, click Refresh button
  2. You will see the device names  you added in the device folder

Here are the examples:

BlackBerry9000: 480x320

LGXENON: 400x240


Nokia 5800: 640x360


Samsung Brooklyn: 320x240


Conclusion
You can easily customize J2ME emulator by modifying the .properties file, even it is really simple, it can improve your productivity considerably.

Further steps
This article is only focused on Wireless Toolkit 2.5.x , Sun released the new wireless toolkit 3.0,  the device emulators are changed a lot. I need to do more investigation on 3.0 emulator in future..

Wednesday, December 30, 2009

Book Review: Apprenticeship Patterns

I finally finished the book “Apprenticeship Patterns – Guidance for the aspiring Software Craftsman”. It took me about 2 months to finish it, because I don’t have enough time to read it, usually I read it at waiting time for the Go train.

This book changes my strategy of my career path. If you want to become a master of software craftsman, then you should focus on improving your craft, not on promotion, not on money. This is a long road.

“sustainable motivations”
You need to keep motivate yourself, nurture your passion, beware don’t let yourself trapped into Golden Lock:" I'd like to learn something new ,but what I already know pays too well."

You need to find a better company, better job, team and co-worker. Love your job, do what your love, then money will follow. Money itself is not the goal, it is just the byproduct along the road to master. It gives me an insight that you should focus on finding the job that you truly love and passionate, even if it is low pay, because once you have the passion, you will generate creativity and energy, then it will generate the money for you. Another great example is Steve Jobs, in his commencement address to Standford’s class of 2005, he said: “You've got to find what you love “..., “Don't let the noise of others' opinions drown out your own inner voice. And most important, have the courage to follow your heart and intuition. They somehow already know what you truly want to become. Everything else is secondary.”

This book helps me reinforce the understanding of the patterns, the author draws quite a few maps, connecting different patterns, then I realize the importance of the map, because it reveals the connections and relationship of each pattern. It inspire me that pattern itself is important, but also the connections. We should not only focus on each pattern itself, but also their relationship to other patterns.
Those patterns seems simple, it give ms an insight that yourself can find your own pattern if you think deliberately.

My favorite patterns in the book are:
  • Be the worst
  • The white belt
  • Nurture your passion
My personal goal is finding the Tao of programming, which is the right way of software developing, so go from apprentice to master is a must path for me.

I have another blog about this book: My personal practice map.

Tuesday, December 22, 2009

The Evil of Java ME preprocess

Developing Java ME application is painful, because of Java ME's own fragmentation in nature. Java Mobile developers have to deal with different devices, and each device or platform has different implementation. Writing a portable application to support different devices is really hard, and some time it is really a torture.

To solve the fragment issue, NetBeans IDE provides a solution - Preprocess, it is similar to pre-compile in C and C++. Actually preprocess in NetBeans is quite perfect, quite powerful. When I first use it, I feel it is so powerful, so easy to use.  Several years later, when I look at our code, which have #if, #else every where, I just feel the code are so ugly, seems to return to the c code era. I really feel preprocess is an evil! Following are my reasons.

Violate OO Principles
Since preprocess is so easy to use, just add something like comments, it gives developer a backdoor to break OO principles easily and brutally. It is so easy to hack the program, so there is no need to care about abstraction, no need to care about design patterns. And the problem is my colleagues like to overuse preprocess, mostly even use preprocess for constants!  I saw so many code full of magic numbers, and so many preprocess directive scattered every where to change those values.

It also violates OCP principle, OCP means "Open for extension, closed for modification". But preprocess is modifying the existing source code when you add new preprocess configuration. It is quite common that the change works for the new configuration, but will break the old ones.

I found if we use more design patterns deliberately, like strategy pattern, abstract factory pattern etc, and use constant class instead of magic number directly, we can minimize the preprocess greatly.

Hard to Refactor
Refactoring the preprocessed code is really difficult, for example when you rename a method name or variable name, you can only refactor the current active preprocessed code, because others are commented out. Right now in my current project I have 4 different devices type, so every time when I refactor, I have to test 4 times to make sure my change is working. It is really painful!

Hard to Reuse
If you want to reuse a class or a component to another project, but when you see so many preprocess scattered in the code, probably you will not use it.  Either you have to add all those preprocess into your new project which might does not make any sense, or you have to manually remove them, which is even harder to test.

Hard to understand
When you read the code has so many #if #else statement, it is ugly to read, more difficult to understand, code is not clean.

Hard to maintain
When you want to port the current application to a new device, you have to understand how the previous preprocess work; when you want to fix a bug for specific device, you have to make sure it won¡¯t break the others. Manage so many preprocess configuration is really painful. So preprocess is good for short-term, but really painful for long term.

IDE Lock-in
Because of widely use of preprocess, our projects are locked in NetBeans IDE, we can not use other IDE, like Eclipse.  For my experience, NetBeans IDE has some problems for example:
  • you can't use different source code folder for different configuration, you can easily do that in Eclipse MTJ;
  • Build automation is difficult, I don't understand the NetBeans generated build xml, I have to build inside the IDE, I never tried to run build file outside the IDE, and I don't know how create a customized ant build file, I think it is really hard;
  • Difficult for team work. NetBeans project use absolute path, and private project properties. Which means it works in your computer, but might not work in your co-workers environment, or you have to take a while to set up the environment correctly.

Conclusions
Java ME preprocess is easy for short-term, but painful for long term, it solves one issue by generating more issues in future; it delay your pain in a short term, but you will be more painful for long term.

Suggestions for anti-preprocess
  • Try to minimize the use of preprocess, we can't 100% percent remove the preprocess, but be ware to use it, only use it when no other options.
  • Have the discipline of writing clean code, readable code and maintainable code, before you use preprocess, think it carefully, if there is better solution;
  • Use OO design patterns, for example strategy pattern and abstract factory can remove lots of preprocess;
  • Try to use configuration file instead of preprocess, if we put the constant value into resource property file instead of hardcoded using preprocess, we can also make the code cleaner;
  • Centralize the preprocess, if you have to use preprocess, don't allow them scattering every where in the code, try to put those preprocess specific logic together, even better to put them into one class, it would be easier to read, understand and maintain;
  • DSL/Meta programming, if fragmentation is unavoidable, why not use code generation technology to generate source code for each specific platform for us? This needs compiler skills and Domain Specific Language knowledge, it is really an interesting topic.

Saturday, December 12, 2009

My Personal Practices Map

Currently I am reading the book called Apprenticeship Patterns – Guidance for the Aspring Software Craftsman. In page 85, the pattern “Reflect As You Work”, it mentions a technology called “Personal Practices Map”. You can use this technology to analyze and reflect your idea and what you did.
This idea originally comes from London’s Extreme Tuesday Club. The page "Maps of People’s Personal Practices".

This is my Personal Practices Map.



Using map to reflect ideas reminds me the book of GoF design patterns.  In that book, the authors draw a map of all 23 patterns and their relationships.  The book Apprenticeship patterns draw lots of map to connect each pattern.
This is a great technology. In future I will draw more maps:
  • “Draw your own map” (Page 47), draw a map of your career path.
  • Draw the map of reading list

    Saturday, December 5, 2009

    Agile meeting notes at Toronto JUG Dec 1, 2009


    I attended the Toronto JUG meeting on December 1. This time it is about Agile, the presentato is Jack Milunsky from agilebuddy.com. His topic covered agile, SCURM and lean. 
     I put my notes here:

    Adopting agile is hard
      Agile requires both bottom up and top down.
      Applying agile practices is easy, but the hard part is new mindset and new philosophy, which is quite different with traditional waterfall methodology
      Agile/Scrum is pervasive; it involves all different teams to change

    Lean software technology
    Jack spent lots of time on Lean.

    Lean’s two pillars:
    1.      Just-In-Time Flow
    Eliminate inventory,
    Do things in batches, which is similar with SCRUM
    2.      Autonomation:
             Stop the line,
             Built-in reflex

    Just In Time Flow
    -          inventory is waste  => partial done software
    -          Moving in Just in time
    Uncover waste
           Focus on overall concept
           Focus on value

    7 principles of lean software development
    1.      Eliminate waste
    Any partial done software is waste
    2.      Build Quality in
                don’t test it in, prevent defect in the first place
                avoid creating defects
                stop the line, automation, unit test
     3.      create knowledge
               knowledge create process
               don’t lock down
     4.      Defer commitment
                “Planning is good, plans are bad”
     5.      Deliver fast
    Lean focus on time
    Don’t equate fast with hacking
    Expose issues
    Build quality in
    Quality is the first priority
     6.      Respect people
    Self organizing team
    Trust over command and control
     7.      Optimize the whole
    Mindset is important, not just practice
    Transfer ownership
    Focus on the whole value stream, the big picture
    Focus on value, eliminate waste

    Waste

    ·         Extra features
                   Justify every feature, limit features
                   Extra features are waste, > 60% are extra features.
    ·         Relearning
                 Task switching drops the productivity
                  Undocumented code
                  No wiki
    ·         Handoff
             Transition point are waste point, knowledge degrades
              Delays

    ·         Defects

    Increasing speed helps the absence of waste

            Cycle Time =  things in process / average completion rate
    How to reduce the cycle time:
    -          minimize the number of things in process
    -          minimize the size of the things
    -          Recommend small batches, which is similar as sprint in SCRUM

    80% rule: if the task load is over 80%, the cost, defect rate increase dramatically


    Agile in AgileBuddy
    Agile buddy spends 20% time for refactoring, (which is very good, they know how important refactoring is, they do it deliberately, very nice!)
    Discipline:  set the dial to 11( 80% rule)
    Sprint length: 2 weeks

    My thoughts
    After the meeting, I talked to Jack, and we exchanged the idea, I believe that SCRUM mainly focuses on management side, it only focuses on what to do, but does not cover how to do, if you look at the SCRUM, there is no TDD, refactoring, pair programming, etc. I always have the question: do they need to be added in the SCRUM sprint practices? How? So I think SCRUM only is not enough, SCRUM should work with XP. So I asked Jack his idea about SCRUM vs. XP. He replied me that SCRUM only definitely not enough, it has to work with XP, because some of the XP practices are really fundamental, like TDD, refactoring. That what they are doing in agilebuddy.

    I also told him my experience that how difficult to persuade my manager to apply agile in my company, how difficult to apply refactoring. He suggest me to organize a lunch and learn meeting, just watch this video by Ken Schwaber about SCRUM.
    I don’t know if it is working. This meeting let think more about agile.Thanks Jack.
     You can visit Jack's blog at here.

    Thursday, November 26, 2009

    Code Kata: SuDoKu Solver

    This week I am interested in the Sudoku puzzle, I am thinking how to use program to solve the SuDoKu. I almost spent a week for it. At first I was inspired the a book called “Wicked Cool Ruby Scripts”in chapter 5, it provides a ruby source code for the SuDoKu solver. I spent two days to understand the code. The problem is the book does not describe the algorithm in details, and I don’t know how to debug the ruby code in TextMate. So it took me a longer time to finally understand the whole logic.

    Actually it uses the brutal backtrack method. Try fill a number in one cell, then recursively call itself, if it works, continue, then return, try another, until it fills all the 81 cells. I made some changes based on the original code, the major changes once the solver find the solution then exit the loop, while in the original code the program will continue loop until the end, I compare my changes with the original code, the total times executed and execution time both drops about 50%.

    Here is my modified code:

    class SudokuSolver

    def initialize(puzzle)
    @@p = puzzle.split(//)
    end

    #Algorithm to solve the sudoku puzzle
    def solver

    #81 times for each block in the puzzle
    81.times do |j|
    next if @@p[j].to_i!=0

    candidatesArray = getCandidates(j)

    candidatesArray.each do |v|
    @@p[j]=v.to_s
    foundAll = solver
    if(foundAll == true)
    return true
    end
    end

    # finished trying all the candidates, backtrack
    @@p[j]=0
    return false
    end

    return true
    end

    def getCandidates(j)

    Array array = [1,2,3,4,5,6,7,8,9]

    row = j/9 *9
    col = j%9

    #find the cells with the same row
    9.times do |k|
    m = @@p[row+k].to_i
    if(m)
    array.delete(m)
    end
    end

    #find the cells in the same column
    9.times do |k|
    m = @@p[k*9 + col].to_i
    if(m)
    array.delete(m)
    end
    end

    #find the cells in the same sub cube
    start_row = j/27 * 3
    start_col = j%9/3 * 3
    3.times do |t|
    3.times do |l|
    m = @@p[(start_row + t)*9 + start_col + l].to_i
    if(m)
    array.delete(m)
    end
    end
    end

    return array

    end

    def printResult
    puts "\n\nThe Solution is:\n"
    print "+-----------------------------+\n|"
    1.upto(81) do |x|
    print " #{@@p[x-1]} "
    if x%3==0 and x%9 !=0
    print "|"
    end
    if x%9==0 and x%81 !=0
    print"|\n|-----------------------------|\n|"
    end
    if x%81==0
    puts "|"
    end
    end
    puts "+-----------------------------+"

    end
    end


    question = "000700390090500000300240800700900200000000000003007008004026007000005060026001000"
    answer = SudokuSolver.new(question)
    puts "\n\n\nSolving puzzle, wait one moment..."
    answer.solver
    answer.printResult

    The logic is really simple: first reads the puzzle, construct the one dimension array.
    The key is in Solver method:
    From 81 cells, pick one empty cell, calculate the candidate numbers of this cell,
    Pick one number, fill the cell,
    Recusively call itself, continue try to fill another empty cell.
    If the program could not find the candidate for the cell, or the program iterate all the cell’s candidate number, then erase the number, backtrack;
    If the program fills all the 81 cells, return true

    The algorithm of getCanidates(j) is easy:
    Build an array of 1-9
    Find each cell with the same row, erase the number in the cell from the array;
    Find each cell with the same column, erase the number in the cell from the array;
    Find each cell with the same sub block, erase the number in the cell from the array.
    The left in the array are candidates


    From this code, I understand more about the backtrack and recursive method. This code belongs to the depth-first backtrack search.

    Donald Knuth’s Dancing Link algorithm is quite interesting, there are lots of articles and blog about using it solving the SuDoKu. But right now for me it is really hard to understand.

    For SuDoKu, there are following thins to do in future:
    1. Implement the SuDoKu solver using Dancing Link algorithm
    2. Find out the algorithm of SuDoKu Generator, like how to generate a puzzle based on difficulty, and how to find out there is only one solution.
    3. How to implement an algorithm that simulate human's solving logic?

    Here are several link about dancing link and SoDuKu:
    Dancing Link C implementation
    Dancing link for Java
    Donald Knuth Home page
    Donald Knuth's Dancing links paper(pdf)