Real Device Vs Emulator Testing: Ultimate Showdown

Real Testing Device:  Testing on real device allows you to run your mobile applications and checks its functionality. Real device testing assures you that your application will work smoothly in customer handsets.

Emulators: Emulator is a software program that allows your mobile to imitate the features of another computer or mobile software you want them to imitate by installing them to your computer or Mobile.

Difference between the emulator and simulator based testing:

Both Emulators and Simulators are virtual devices. A virtual device is not the real phone but software which gives same functionality as the real phone (except few functionality like the camera).

The simulator based testing	The emulator based testing
Simulator's objective is to simulate the internal state of an object as close as possible to the internal state of an object.	The emulator aims at emulating or mimicking as close as possible the outer behavior of an object
Simulators are preferable whenever the testing team needs to test the mobile's external behavior like calculating, making transactions and so forth.	Emulators are preferable whenever the testing team needs to test the mobile's internal behavior like its internal hardware, firmware and so forth.
Simulators are written in high level languages.	Emulators are written in machine-level assembly languages.
The simulators can be difficult in terms of debugging purpose.	Emulators are more suitable when it comes to debugging purpose
A simulator is just a partial re-implementation of the original software .	Often an emulator comes as a complete re-implementation of the original software .

Relative Advantages of real device-based application and emulator/simulator based testing

Issue	Emulator Testing	Real Device Testing
Situation-based application	There are specific situations where the deadline to produce text execution results are short and purchasing the required mobile devices may be not possible. Thereby it might be necessary to use the emulator/simulator in these circumstances for testing the relevant mobile applications which need to be tested.	The real device allows the testers to test almost all the real time scenarios which can be tested for the mobile applications. These devices are operated using fingers and simulate real-life usage. They also help in situation Real context: is it easy to use the app on the train, or while walking down the street? The situation about in bright sunlight or in the rain?
Feeling of closeness towards the real handheld devices	The wide gamut of mobile devices creates the problems, whereby the testers are not confident about which mobile devices to invest in for testing, considering the budget constraints. Emulator/simulator (s) is tailor made for this type of situation(s).	The real device allows the testers to test even usability issues like the look and feel of the application, color resolution of the screen, whether the picture is bright or not under both day and night conditions and so on.
Ease of availability	Emulator/simulator(s) are in most cases open and free software which can be very easily downloaded from Internet and ready to be tested for.	The real devices allow stringent performance testing issues like working with a real time transport application for 15 hours continuously which cannot be successfully simulated by the emulators.
Ease of opening an Web application through URL	It is easier to do web application testing when it comes to opening the web application. The user just needs to copy and paste the application URL.	Testing on real devices provides more in terms of reliability.
Capturing screenshots of the situations where defects appear	Capturing issue of screenshots over simulator is very easy with the simulator since we just need to use Microsoft office facilities.	Testing with real world devices is very helpful in terms of interoperability testing.
Simulation of validation of battery scenarios	The emulator/simulators are not able to simulate the battery issues.	Real world devices can easily perform the same.
Validation of incoming interrupts	The emulator/simulators are not able to simulate the incoming interrupts for SMS as well as the incoming calls.	Real world devices can easily simulates incoming interrupts.
Validation of exact color displays	The emulator/simulator is not able to properly emulate/simulate the exact color display of the devices when the real device is in sunlight or in black.	Real world devices can easily simulates the exact color displays.
Validation of the performance	The performance of the emulator/simulator tends to be slower than the original devices at times.	The original devices tend to perform faster than the emulator or the simulators.
Simulating memory related issues	The memory available at the emulator/simulator tends to be far more than the real devices so this may create misconception for the users who would be using the same validations.	The memory storage level of the devices tend to be far less than the emulators thus it may

Disadvantages of Simulators and Real device

Emulators/ Simulators	Real Device
The emulator/simulator is not always the best type of solution for scenarios such as the ones whereby the testing team needs to validate the performance of the application for a longer period of time.	The real devices are costly compared to the emulator/simulators. Thereby projects under budget and timeline constraints may risk profitability as well as the viability of the overall project.
The emulator/simulator is suitable mostly for certain types of functional test case executions.	There is a very wide variety of mobile devices from apple to Samsung to android and to Symbian and so on. Considering this wide range of mobile devices it is very hard for the testing team to arrange all sorts of mobile devices while working under considerable amount of budget and timeline related constraints.
The emulator/simulator can some time not be supportive of certain types of application and in these cases the testing team may need to purchase software patches which may not always be free but could be costly at times.	Real Mobile devices when used in the developing stage for unit testing and similar purposes could turn out to be harder to connect to the IDE than the emulators and this causes tremendous problems for debugging and in a project with timeline constraints this may very well hamper the overall conclusion of the project.
Not all the emulator/simulator supports the complete gamut of mobile applications. For example the bada simulator supports the Maemo (such as Nokia N900), Symbian Touch (such as Nokia N8) and Symbian non-touch (such as Nokia E71) but it does not support other mobile devices like Android. As per as the application testing functionalities are concerned, bada does not support direct web browsing testing but it allows the user to test and create only webapps and widgets.	In order to test with the real world devices, the devices need to be always connected to the USB port of the machines. So if the USB ports are not working properly, the testing would not be possible. Without providing adequate security measures mobile devices (if they happen to be costly like the apple Iphone) may be lost or stolen thus hampering the overall effort. Increasing security may also go on to increase the overall expenditure involved with the project.
	The user has to type manually the URL for opening up the web application which is needed to be tested. To solve this particular issue, the tester may need to create phone bookmarks, short URL services or sending URLs to mobile using Bluetooth connection or creating the webpage that contains some URL-s. The adoption of these procedures would ensure that a lot of very important memory space may be eaten up thus impacting on the overall performance of the application.

Conclusion

Considering the significant role the mobile applications is playing nowadays play now days in our day to day life, testing of these applications are going to evolve and thus they require a lot of testing to make them work as required.Testing in both the simulator/emulator as well as the real world devices is necessary to maintain strong standards and quality assurance.

Careful deliberation of both the pros and cons of mobile emulators and real devices, it would be worthwhile to reach at the conclusion that the optimal mobile testing solution for enterprises is neither putting all the eggs into the basket of the real devices nor putting them into the emulator but rather what we need is an optimum combination of both.

Emulators can be considered as very suitable for the initial stages of application development.

However, to avoid the costly scenario of releasing a business-critical application with defects, enterprises need to ensure that they perform the major part of their mobile testing on real devices before the application goes into production.

Each organization needs to strategize and plan carefully to determine at what stage to introduce real devices, they also need to decide how many devices are sufficient to cover market needs, and what could be the best possible option to adopt for managing those devices.

Best practices would indicate that actual development should use emulators (and a few reference real handsets) in order to speed up the debugging of the application during the coding phase, while sanity, smoke testing, performance, interoperability and network feasibility and regression testing should be done on real handsets.

It is also an emerging practice to ensure that the developers use the emulator for fast execution during the development phase whereas then the testing team should test with the real device during the testing phase in order to ensure overall quality assurance goals and targets.To save on cost, they can consider using Virtual Mobile Testing tools. These services offer developer to test their application on wide variety of handsets using different mobile networks geographically located throughout the world (useful for applications using GPS). Such services are offered on hourly basis and are very cost effective compared to buying new phones.