Imitating a Browser

There are several solution that QA team is using to simulate browser behavior to doing load/performance tests. I was just looking at HTMLUnit (http://htmlunit.sourceforge.net) for a internal test of browser. It is simple and have all basic features that we generally need to test a browser based application.

Code can be configured to test with multiple browsers. I have tried a simple program in java to test it, it is great!! Look at above site for more information.

package com.test;

import com.gargoylesoftware.htmlunit.FailingHttpStatusCodeException;

import com.gargoylesoftware.htmlunit.WebClient;

import com.gargoylesoftware.htmlunit.html.HtmlForm;

import com.gargoylesoftware.htmlunit.html.HtmlImageInput;

import com.gargoylesoftware.htmlunit.html.HtmlPage;

import com.gargoylesoftware.htmlunit.html.HtmlPasswordInput;

import com.gargoylesoftware.htmlunit.html.HtmlTextInput;

import com.gargoylesoftware.htmlunit.util.Cookie;

import java.io.IOException;

import java.net.URL;

import java.util.Iterator;

import java.util.Set;

/**

* Invoke any identified HTML files with out browser and gether result from it.

*/

public class HTMLInvoker {

private static String url = “your url”;

public static void main(String[] args) {

WebClient webClient = new WebClient();

HtmlPage page = null;

try {

URL siteUrl = new URL(url);

page = (HtmlPage) webClient.getPage(siteUrl);

HtmlForm form = page.getFormByName(“form”); // View source of page and check form name

HtmlTextInput id = (HtmlTextInput) form.getInputByName(“id”); // View source and check id/name of element/textfields

HtmlPasswordInput passwd = (HtmlPasswordInput) form.getInputByName(“password”); // View source and check name/id of password field

HtmlImageInput submitButton = (HtmlImageInput) page.getElementById(“submit”); // View page source and get button id/name

id.setValueAttribute(“arvind@gmail.com”); // Set value to text field

passwd.setValueAttribute(“arvind123”); // Set password field

webClient.getOptions().setThrowExceptionOnScriptError(false); // Avoid script validation

submitButton.click(); // Submit your page

}

catch (FailingHttpStatusCodeException e1) {

// TODO Auto-generated catch block

e1.printStackTrace();

}

catch (IOException e1) {

// TODO Auto-generated catch block

e1.printStackTrace();

}

// RESULT: Process your results

Set<Cookie> cookie = webClient.getCookieManager().getCookies();

// Here several objects related to page can be fetched.

if (cookie != null) {

Iterator<Cookie> i = cookie.iterator();

while (i.hasNext()) {

Cookie ck = i.next();

if (ck.getName().contains(“any_cookie”)) {

String cookieValue = ck.getValue();

System.out.println(“Cookie value:” + cookieValue);

break;

}

}

}

webClient.closeAllWindows();

}

}

Generating TCP Trace

TCP trace helps solving several issues whether application logger is enabled or not. These days when applications are becoming more and more service oriented, finding issue in between layers is tricky and cumbersome. TCP tracing helps to compare request and response of success and failed TCP traces to identify issues in network calls. Primarily my communication protocol is HTTP, however traces can be generated for any protocol.

Tool 1: Grinder TCP Trace;

Grinder is generally used for load testing however it is quite handy for traffic routing and TCP proxy. More information is provided here.

Just download Grinder proxy in any folder and create a sample script having following command to quick start your proxy:

java -cp grinder/grinder.jar -Xms16m -Xmx32m net.grinder.TCPProxy -localhost localhost -localport 9090 -remotehost $1 -remoteport $2

Now, just start your proxy using following simple command. You need to provide remote host and remote port information to script.

 ./grinder.sh tesserver.com 8080

Now proxy is started on localhost:9090. Just replace your real URL with proxied URL to get TCP trace on command prompt (Terminal). Ex: http://localhost:9090/index.html -> will call http://tesserver.com:8080/index.html

Tool 2: Wireshark

Wireshark does not need any introduction, it is great tool to generate trace for any communication and quite common in software/hardware industry.

Wireshark comes with installation for most of platform. However installing on MAC needs few more steps:

– Download Wireshark from here.

– It needs X windows system on some OSX. For that you can download XQuartz from here.

– Simple logout/login after installation of xQuartz.

– Start wireshark after installation. It will ask for X11 installation. Browse your folder to “/Applications/Utilities/XQuartz.app/Contents/MacOS/X11” application.

– It may open X11 application with Wireshark, close both and restart Wireshark again. Great, we are good to go next.

TCP/HTTP traces are easy with Wireshark. There are plenty of information available on internet. For me, following traces are enough:

– Start Wireshark traces with identified network trace (Select Interfaces list and identify your available networks)

Screen Shot 2015-03-09 at 5.08.47 PM

– Now, Wireshark will generate lots of traces, just select HTTP text from ‘Filter’ text box:

Screen Shot 2015-03-09 at 5.11.49 PM

– Right click on HTTP request and select ‘Follow TCP Traces”

Screen Shot 2015-03-09 at 5.14.38 PM

– Your TCP trace are zoomed out. It can be saved in your disk.

Screen Shot 2015-03-09 at 5.16.06 PM

Cheers :)

Eclipse – Data Assist in tooltip

While debugging in eclipse, hovering is not showing the real data (value) of parameter and annoying developer by showing method type. Hence developer needs to go debug screen (or right click -> Display) to see value of a variable while in debug mode.

I need to see value on hovering mouse on a variable rather than right click or change eclipse perspective.

For that: Go to Preferences -> Search Hover: You will see following default screen:

Screen Shot 2015-02-16 at 11.54.26 AM

Now un-select “Combined Hover” option and clean out “Source” option. Remember you need to keep “Source” option selected. Apply and Save.

Screen Shot 2015-02-16 at 11.56.11 AM

Great, now you can see respective value of a variable in tool tip. Also if you hover on a method, you can see detail implementation of method.

Setting Java Cryptography Extension (JCE) Unlimited Strength

Default java installation is not coming with high encryption extension files. These files are valid for USA country and are need to be downloaded and installed separately:

1. Download the JCE Policy jar files from the below location:

http://www.oracle.com/technetwork/java/javase/downloads/jce-7-download-432124.html

2. The zip file would contain two jar files (local_policy.jar and US_export_policy.jar).

3. These jar files need to be placed under the ‘jre/lib/security’ location. For my MAC, this location is as follow:

/Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/jre/lib/security

Enjoy strong encryption algorithm now :)

Installing Ubantu using VirtualBox

I need to have separate VM to test my active-active use cases, that need multiple VMs to simulate multiple data centers requirement.

Installing Oracle Virtual box and ubantu is quite simple, though there is minor issue in configuring network and resolution.

Just note down steps:

    • Download Ubuntu (64 bit) from “http://www.ubuntu.com/download/desktop&#8221;, Save .iso’ image file in your disk.
    • Download Oracle VirtualBox from virtualBox.org
    • Most of configuration are intuitive and can be re-configured later. Select default setting.
    • On start of VM, it will ask for operating System, browse to downloaded image (.iso) file.
    • Start VM and install Ubantu from stating menu
    • Select “something else” on stating process
    • While installation, create ext4 partition. Select approx 7GB space allocation (less than memory available) and mount on root (“/”) directory.
    • Type ALT+Ctrl+T to get command promt
    • sudo apt-get install default-jdk (It install open JDK version 7u71). You can install other versions as per requirement of project
    • To configure networks,
      • select ‘network’ on Virtaulbox panel > Adaptor 1 > Bridge Adaptor > xxxx Gigsbit Network Connection Card   (for eathernet)
      • Again Adaptor 1 > Bridge Adaptor > xxx centrino Advanced (Wireless)
    • Chage user to ‘root’ by ‘sudo -i”
    • gedit /etc/network/interfaces
    • Add following to DHCP

auto eth0
iface eth0 inet dhcp

    • Add following for static

auto eth0
iface eth0 inet static
address 10.253.0.50
netmask 255.255.255.0
network 10.253.0.0
gateway 10.253.0.1
dns-nameservers 8.8.8.8

    • Release and reconnect connection to eth0

      sudo ifdown eth0

      sudo ifup eth0

    • sudo service resolvconf restart
    • Update resolution of screen

sudo apt-get install virtualbox-guest-dkms

    • restart Your VM
    • Install Tomcat, if there is a need

wget http://mirror.atlanticmetro.net/apache/tomcat/tomcat-7/v7.0.29/bin/apache-tomcat-7.0.29.tar.gz

    • Configure .bashrc for java and tomcat path

sudo nano ~/.bashrc

Done.

SSL enabled TCP Trace

I was facing an issue while taking TCP trace on client end for SSL enabled server, encrypted trace :)

I like using Grinder for TCP dump by running TCP Proxy on different port.

Following is the way, you can generate readable (decrypt) trace:

1. Using Grinder:

java -cp grinder/grinder.jar -Xms16m -Xmx32m net.grinder.TCPProxy -localhost localhost -localport 9090 -remotehost  <Server Address> -remoteport 443 -keystore myserverJKS.jks -keyStorePassword abcd1234 -ssl

Run above command out of your grinder package (or change jar file path accordingly)

Note “-ssl” keyword in command.

2. Using Wireshark

Wireshark is great and can easily be configured for SSL based TCP traces:

– If you do not have .key file, regenerate it from your JKS and password.

keytool -certreq -alias <domain-name> -keystore <jks file name> -file myKey.key

It will ask your jks password, it should be same as it is used to generate JKS.

– Open Wireshark, Edit > Preferences > Protocols > SSL > (Pre)-Master-Secret log filename > select .key file

– Start TCP trace

Use efficient steaming to upload your files to server

I was trying to figure out if I can upload files using streaming process.

What is streaming process: byte by byte upload.

Advantage: On interruption on upload process due to bad/slow network connection, you do not need to send bytes again that has already been sent to server/storage disk.

Disadvantage: You need to send each byte.

Now let’s choose the middle way, by using buffers. We can load buffers with bytes and send to storage/server when buffers are full.

Let’s see the attached code:

final byte[] bytesRead = new byte[bufferSize];
int noOfBytesRead = 0;
long totalNoOfBytesRead = 0;
long endOffset = 0;
long endOffset1 = 0;
long beginOffset = startingOffset; // if we have any starting offset, else start with 0
final ByteArrayOutputStream baos = new ByteArrayOutputStream(bufferSize);

while ((noOfBytesRead = stream.read(bytesRead)) != -1) {
endOffset = totalNoOfBytesRead – 1;

final byte[] bytesToStorage = new byte[noOfBytesRead];

System.arraycopy(bytesRead, 0, bytesToStorage, 0, noOfBytesRead);

if (baos.toByteArray().length > bufferSize) {
endOffset1 = (beginOffset + baos.toByteArray().length) – 1;
}
uploadBytes(baos.toByteArray(), beginOffset, endOffset1);
beginOffset = endOffset1 + 1;
baos.reset();
}
baos.write(bytesToStorage, 0, bytesToStorage.length);
}

}

endOffset = endOffset + startingOffset; // If there is starting offset, else no need.

if ((baos.toByteArray().length != 0) && (baos.toByteArray().length <= bufferSize)) {
uploadBytes(baos.toByteArray(), beginOffset, endOffset);
baos.reset();
}

Simple use of buffer: Few things to remember-

  1. Keep reading your bytes unless we have reached to end of file.
  2. Load data in buffer, for that define ByteArrayOutputStream to copy data from input stream.
  3. Calculate begin and end offset positions as we are moving bytes along
  4. Keep checking buffer size, if is more than defined limit, send all bytes to server (storage) with start and end offset and clean buffer.
  5. In last section (After while loop), if we have reached to end of file (bytes == -1), send remaining bytes to server/storage with correct starting and end offset.

Above method can be optimized a bit, but it works well for us, hence I keep myself stick to it.

Follow

Get every new post delivered to your Inbox.