Road to integration of calabash android test with Jenkins

In this post I am going to show you how to integrate calabash android test with Jenkins continuous integration for build. Also I will show you how to publish cucumber execution report in Jenkins.

Installation:
1. Java: java should be installed
2. Jenkins: If you are new to Jenkins then click link "click here” of my post how to setup Jenkins in window before continue.
3. Install the”cucumber-jvm-reports-java” plugin: following are the steps to install plugin.
  • Go to dashboard of Jenkins.
  • Click on “Manage Jenkins”
  • Click on “manage plugins”
  • Click on “Available” tab
  • Search “cucumber-reports” and install plugin
  • Restart Jenkins server.

Calabash android with ant: for the integration of calabash android test with Jenkins we need to create ant build file for the test execution. Below is the ant build file for the calabash android test.

Here we are using junit json report. cucumber-jvm runs and publish json format report into html format
--format json -o cucumber.json  


Jenkins Project: following are the steps to create calabash android Jenkins project.
  • Go to Jenkins dashboard.
  • Click on “New Jobs”
  • Enter Jobs name and check “Build a free-style software project” check box.
  • Click on "OK" button. 
  • Click on “Add build step” options under build and click “Invoke Ant” option.
  • Enter your target name in “Targets” filed and enter build file name with path into “Build File” field.
  • Click on “Add post build action” and click on “Publish cucumber results as a report” 
  • Click on save button.
  • Leave Json reports path blank to find automatically report. If automatically not read json file then you enter path name.

Reports:

After build execution click on “Cucumber Reports” link your execution report is generated as below.





39 comments:

  1. Thanks really this is very nice post for calabash integration with Jenkins I was looking from couple of days. Now i am able integrate Calabash with Jenkins.

    Thanks again for this post...

    ReplyDelete
  2. thanks, Nice post for integration .
    keep it up .....


    ReplyDelete
  3. I added the above code in my build.xml ant script as follows:








    But when I build my project in Jenkins, I get the following message:
    BUILD SUCCESSFUL
    Total time: 1 minute 8 seconds
    $ D:\adt-bundle-windows-x86\sdk/platform-tools/adb.exe disconnect localhost:2535
    [android] Stopping Android emulator
    [android] Archiving emulator log
    $ D:\adt-bundle-windows-x86\sdk/platform-tools/adb.exe kill-server
    [CucumberReportPublisher] Compiling Cucumber Html Reports ...
    [CucumberReportPublisher] detected this build is running on the master
    [CucumberReportPublisher] there were no json results found in: C:\.jenkins\jobs\igLogisticsTest\workspace
    [CucumberReportPublisher] there were no json results found in: C:\.jenkins\jobs\igLogisticsTest\builds\2013-08-08_14-13-44\cucumber-html-reports
    Finished: SUCCESS


    Please help me running the calabash-android scripts through Jenkins and generating the report for the same.

    ReplyDelete
  4. What error you getting in report file. Did your calabash text executed or not ?

    ReplyDelete
  5. No error is shown. Apk gets installed and then the emulator is stopped. I can't see my feature files getting executed. Am I missing something?

    ReplyDelete
  6. Are you able to run using command line without build.xml file. Just using calabash-android command ?

    ReplyDelete
  7. Yes, am able to run calabash-android using command prompt (without build.xml)

    ReplyDelete
    Replies
    1. Make sure you are passing your scenario name instead of @LoginApp after --tags

      Delete
    2. I did not pass any scenario name. I am running all my features as:

      Delete
    3. Can you share your build.xml file?

      Delete
    4. This comment has been removed by the author.

      Delete
  8. This comment has been removed by the author.

    ReplyDelete
  9. Please email me on roadtoautomation@gmail.com

    ReplyDelete
  10. Even i am facing the same issue

    ReplyDelete
  11. Hi,
    I need your help to integrated the calabash-android with jenkins.
    I have made the build.xml file as shown above and have placed it in the workspace folder. But I believe I need to change few parameters according to my project.
    Could you please advice what parameters should be changed.

    Kaushal

    ReplyDelete
    Replies
    1. What type of parameter you want change ?

      First run your test scenario by command using ant. After that put all your project under Jenkins workspace and create job as above.

      Delete
  12. Ok let me explain my situation.
    I have created the my calabash-android tests and when I try to run in the command prompt, it runs correctly. In the command prompt, I use the following command
    calabash-android run C:\Mobile\MobileAutomation\Calabash-Tests\SanUK.apk -p none features\my_first.feature --format json -o cucumber.json
    Now I want to run these calabash tests in Jenkins. I read your blog but got a bit confused as this is my first attempt to work with Jenkins.
    I have installed Jenkins and the cucumber-reports plugin.
    Now as per your blog, I tried to create the ant build file by copying all the contents shown in your blog. I have put this build.xml file in the below location
    C:\Users\c0209693\.jenkins\jobs\Android_Native\workspace
    But in the build.xml file could you please explain what should I change the value of these tags?
    Project name
    Default
    Basedir
    Property name & location  location where the apk file is kept
    Target name
    Exec dir
    Executable = cmd
    Arg line

    As of now I have copied all the values as mentioned in your blog and configured the job as per your instructions. Now when I build it, I get the following output

    Started by user anonymous
    Building in workspace C:\Users\c0209693\.jenkins\jobs\Android_Native\workspace
    [workspace] $ cmd.exe /C '"ant.bat -file build.xml Login1 && exit %%ERRORLEVEL%%"'
    Buildfile: C:\Users\c0209693\.jenkins\jobs\Android_Native\workspace\build.xml

    Login1:
    [exec] *** WARNING: You must use ANSICON 1.31 or higher (https://github.com/adoxa/ansicon/) to get coloured output on Windows
    [exec] cucumber.yml was not found. Current directory is C:/Users/c0209693/.jenkins/jobs/Android_Native/workspace. Please refer to cucumber's documentation on defining profiles in cucumber.yml. You must define a 'default' profile to use the cucumber command without any arguments.
    [exec] Type 'cucumber --help' for usage.

    BUILD SUCCESSFUL
    Total time: 4 seconds
    [CucumberReportPublisher] Compiling Cucumber Html Reports ...
    [CucumberReportPublisher] detected this build is running on the master
    [CucumberReportPublisher] there were no json results found in: C:\Users\c0209693\.jenkins\jobs\Android_Native\workspace
    [CucumberReportPublisher] there were no json results found in: C:\Users\c0209693\.jenkins\jobs\Android_Native\builds\2013-10-11_00-01-32\cucumber-html-reports
    Finished: SUCCESS

    my build.xml file is this:










    Right now in my workspace folder there is only one file - build.xml

    Please guide me step step how to resolve this issue

    ReplyDelete
    Replies
    1. did you put only build.xml file or all project into workspace ?
      As per your error I observed that you only put build.xml file into workspace.

      Please put all your project in workspace with build.xml file.

      Delete
  13. Hi,
    I have one more query.
    How can I link Jenkins with ALM 11 to run the calabash scripts.

    I have installed the "HP Application Automation Tools" and have configured the job accordingly. But I am getting the below error when running the job.

    Failed login to ALM Server URL: http://alm3........gs.corp/qcbin/start_a.jsp. Exception: java.io.FileNotFoundException: http://alm3........gs.corp/qcbin/start_a.jsp/rest/is-authenticated
    Empty Results
    Result Status: UNSTABLE
    Build step 'Execute HP tests using HP ALM Lab Management' changed build result to UNSTABLE
    Report not found
    Build step 'Publish HP tests result' changed build result to FAILURE
    Finished: FAILURE


    Please help.

    ReplyDelete
    Replies
    1. I have no experience on Jenkins with ALM 11. I will reply if I get any solution.

      Delete
  14. Is anyy way to use cucumber report plugin if i pass the json report file and it will generate the html report using cmd prompt

    ReplyDelete
    Replies
    1. No need to generate report in JSON format, You can pass flag "-f html -o test-result.html" with calabash run command to generate report in HTML format.

      For more detail go below post:
      http://roadtoautomation.blogspot.in/2013/06/road-to-create-calabash-android-test.html

      Delete
  15. Hi,

    I tried to configure as given above. The build is successful but the report gives 404 error.

    Can you help to resolve the issue.

    I got the following console output:

    BUILD SUCCESSFUL
    Total time: 3 seconds
    [CucumberReportPublisher] Compiling Cucumber Html Reports ...
    [CucumberReportPublisher] detected this build is running on a slave
    [CucumberReportPublisher] copying json from: file:/D:/Jenkins/connections/workspace/Calabash_Android/to reports directory: file:/D:/Jenkins/jobs/Calabash_Android/builds/2014-01-03_16-30-25/cucumber-html-reports/
    [CucumberReportPublisher] there were no json results found in: D:\Jenkins\jobs\Calabash_Android\builds\2014-01-03_16-30-25\cucumber-html-reports
    Finished: SUCCESS

    ReplyDelete
    Replies
    1. are your project and build.xml file into workspace ?

      Delete
    2. Hi,
      Are some one how have got this kind of issues?

      Below show you which fails I usually get when I run a job in Jenkins:
      1: ant file

      [exec] Could not find an Android SDK please make sure it is installed.
      [exec] You can read about how Calabash-Android is searching for an Android SDK and how you can help here:
      [exec] https://github.com/calabash/calabash-android/blob/master/documentation/installation.md#prerequisites
      [exec] Result: 1

      2: shell command

      + calabash-android run /Users/ReinaldoB/calabash-samle-proj/magine/aliceoid_internal_0_36.apk
      Could not find an Android SDK please make sure it is installed.

      If I run above in a terminal all working well. I have got an explication that this issue I got is due to latest version of calabash like 0.4.21.

      Have some of you resolved this issue?

      Delete
    3. I have updated calabash 0.4.21 and getting same issue and tried multiple machine but getting same issue for 0.4.21.
      After downgrading Calabash android my issue resolved

      Delete
  16. Hi,
    I have downgraded Calabash android and the issue is still left. See below.

    [exec] /Library/Ruby/Gems/2.0.0/gems/calabash-android-0.4.9/bin/calabash-android:53:in `raise_if_android_home_not_set': Please set the ANDROID_HOME environment variable (RuntimeError)
    [exec] from /Library/Ruby/Gems/2.0.0/gems/calabash-android-0.4.9/bin/calabash-android:72:in `'
    [exec] from /usr/bin/calabash-android:23:in `load'
    [exec] from /usr/bin/calabash-android:23:in `'
    [exec] Result: 1

    Do you know where in jenkins I need to set the ANDROID_HOME path because in my .profile in mac i have it.

    ReplyDelete
    Replies
    1. I am using calabash-android version 0.4.16 and it is working at my end..

      Delete
  17. Hi,

    I've been struggling with a problem, I don't know how to get over it. Any helps would be much appreciated.
    I'd started with my build.xml configured as shown above on the post. When I started to build, an error occurred like below:

    Started by user anonymous
    Building in workspace C:\Program Files (x86)\Jenkins\jobs\Test\workspace
    [workspace] $ cmd.exe /C '"ant.bat -file build.xml login && exit %%ERRORLEVEL%%"'
    Buildfile: C:\Program Files (x86)\Jenkins\jobs\Test\workspace\build.xml

    login:
    [exec] 'calabash-android' is not recognized as an internal or external command,
    [exec] operable program or batch file.
    [exec] Result: 1

    BUILD SUCCESSFUL
    Total time: 0 seconds
    [CucumberReportPublisher] Compiling Cucumber Html Reports ...
    [CucumberReportPublisher] copying all json files from: C:\Program Files (x86)\Jenkins\jobs\eMobiz\workspace to reports directory: C:\Program Files (x86)\Jenkins\jobs\eMobiz\builds\2014-07-13_10-57-21\cucumber-html-reports
    [CucumberReportPublisher] there were no json results found in: C:\Program Files (x86)\Jenkins\jobs\eMobiz\builds\2014-07-13_10-57-21\cucumber-html-reports
    Finished: SUCCESS

    Thanks for your help,

    ReplyDelete
    Replies
    1. Are you able to run test scripts using CMD, do you have setup ruby bin path in system variable ?

      Delete
    2. Hi,

      Thanks for your quick response.

      My script runs well on CMD, and I also added ruby path into the PATH environment variable.
      I've cloned your build.xml. But I don't know why the system doesn't recognize calabash-android command.
      (In additional, I'm using apache ant 1.9.4, I also added ANT_HOME variable, and ANT_HOME/bin into path env ).
      And I also tried to run cucumber script with the command: cmd c/ cucumber , but it didn't work either.

      Do you have any ideas of this situation?

      Deeply thanks for you help,


      Delete
    3. There might me some thing wrong in setup environments variables. In Environments Variables there are two section one for User variables and other for System variables if you setup path in user variables then remove from there and setup all path in system variables section.

      Delete
    4. Hi,

      I've already placed all environment variables in system variables section.

      I've tried to configure on 2 computers, one works perfectly and another falls into that stuff.

      I think maybe I'm using a higher version of calabash-android. I'll try to downgrade its version and let you know results then. :)

      Thanks for your help,

      Delete
    5. In finally, How you can resolve in this problem please tell

      Thank you.

      Delete
  18. This comment has been removed by the author.

    ReplyDelete

Leave your comments, queries, suggestion I will try to provide solution