Monday, April 27, 2009

Analyzing the memory usage of your Android application


The new Android 1.5 Early Look SDK  is out since a few weeks. The "Android 1.5 highlights" page does not mention one highlight, which IMHO will become very important for all developers on the Android platform because it allows you to find memory leaks easily and analyze the memory usage of your Android applications.

I'm talking about the hprof-conv tool that allows you to convert an Android/Dalvik heap dump into an .hprof heap dump. The tool does not preserve all the information, because Dalvik has some unique features such as cross-process data sharing, that are not available in a "standard" JVM. Still the .hprof file is a good starting point and can be read with the Eclipse Memory Analyzer, which will interpret it like a normal file from a Sun 32 bit JVM. In the future it should also be not that difficult to read the Dalvik heap dump format directly and provide more information for memory analysis, such as which objects are shared and also maybe how much memory is used by native Objects (haven't checked that).

How to get a heap dump

First we must of course get a heap dump. I do not yet own an Android phone, also I plan to get on this year. If you want to donate one, please let me know. Donating lot's of money would also help ;)
I therefore created the heap dump using the emulator.
I created a new avd for the 1.5 preview:

android create avd -n test -t 2
emulator -avd test

Then you can logon to the emulated phone and make sure that the /data/misc directory is writable :

adb shell
chmod 777 /data/misc
exit

Heap dumps are actually created by sending the process a SIGUSR1 (-10) signal.

adb shell ps

check for the pid of your application

adb shell kill -10 pid


You can check with

adb shell logcat

whether it works. You should see something like:

I/dalvikvm( ): threadid=7: reacting to signal 10 I/dalvikvm( ): SIGUSR1 forcing GC and HPROF dump I/dalvikvm( ): hprof: dumping VM heap to "/data/misc/heap-dump-tm- pid.hprof-hptemp". I/dalvikvm( ): hprof: dumping heap strings to "/data/misc/heap-dump-tm124026 3144-pid.hprof". I/dalvikvm( ): hprof: heap dump completed, temp file removed D/dalvikvm( ): GC freed 1292 objects / 92440 bytes in 11 sec D/dalvikvm( ): GC freed 215 objects / 9400 bytes in 963ms

now you can copy the heap dump from the (emulated) phone to the Desktop machine:

adb pull /data/misc/heap-dump-tm-pid.hprof address.hprof

Now the file you will get does not conform to the "standard" Sun .hprof format but is written in Dalvik's own format and you need to convert it:


hprof-conv heap-dump-tm-pid.hprof 4mat.hprof

Now you can use the Eclipse Memory analyzer to analyze the memory usage of your application.

Typical optimization opportunities

I described some typical issues already here in this blog for example a problem with Finalizers in Netbeans,
 or  finding "String duplicates"  in Eclipse (my favourite memory analysis trick).

You might also take the time and check all my memory related posts.

Now I would like to present you a new typical memory usage issue that I just found when playing around with the new Android cupcake 1.5 prerelease.

YouShouldHaveDoneLazyInitialization



I did a heap dump of the android gmail application and loaded it into the Eclipse Memory Analyzer.
In the histogram view I filtered the android classes:



The high number of Rect (Rectangle) objects looked suspicious to me. Also they retain not that much memory, I thought it would be interesting why such a high number of Rect instances was alive.
When checking the Rect objects I immediately saw that most of them seemed to be empty e.g. bottom=0 and left=0 and right=0 and top=0.

It seemed to me that this was the time to again use the most advanced feature in MAT. I'm talking about the OQL view, which allows you to execute SQL like queries against the objects in your heap dump.
I very rarely used this feature in the past because the standard queries are good enough almost all times and because writing your own command in Java is not that difficult.
Still here it would help me to find how many of the Rect instances where empty.

I entered

select * from android.graphics.Rect where bottom=0 and left=0 and right=0 and top=0




Which showed me that out of 1320 Rect instances 941 where "empty", which means that over 70% of the instances where "useless".
I used "immediate dominators" on the first 20 of those 941 instances, which showed me that those empty Rect instances were retained by several UI related
classes which are subclasses of android.graphics.drawable.Drawable.




I checked the source and quickly found that Drawable always instantiates a Rect using the default constructor:



public abstract class Drawable {    private int[] mStateSet = StateSet.WILD_CARD;    private int mLevel = 0;    private int mChangingConfigurations = 0;    private Rect mBounds = new Rect();


Also this issue might not increase the memory usage of gmail that much it's still a fundamental/systematic problem, which IMHO should be fixed.
Even if we would tolerate the memory overhead, we are still left with the problem that the number of objects the garbage collector has to trace during Full GC is higher than needed, which at least potentially can lead to sluggish UI.
It's also a kind of fundamental problem because all subclasses of Drawable inherit the problem.

So instead of always using the default constructor it would most likely better to use lazy initalization  and only initialize the field when it's first needed.
A copy on write strategy  could also be used by sharing an "empty" Rect instance in the beginning, which would be replaced by a new one as soon as the field is written ( haven't checked the code whether this can still be done).

Disclaimer:
This is not a sign that Android is a poor platform. This is a kind of a problem that I've seen in the past more than once and I'm sure this kind of antipattern can be found in a lot of software out there.

UPDATE:


The issue has been fixed a few hours (or minutes?) after my post!
Check
https://review.source.android.com/Gerrit#patch,sidebyside,9684,2,graphics/java/android/graphics/drawable/Drawable.java

To see how simple the change was!


Just take the Eclipse Memory Analyzer and have a look at your application(s).
May the force be with you! ;)

Update 2:
In Android 2.3 (Gingerbread) a new command is available to trigger the heap dump.
The kill command is not supported anymore.




34 comments:

Anonymous said...

Here's a patch:

https://review.source.android.com/Gerrit#change,9684

Unknown said...

Great!
That was really quick.
Looks good to me as well :)

harish said...

Great post.Thanks Markus.

aayush said...

We can also use The traceview tool for memory and performance profiling:

http://developer.android.com/guide/developing/tools/traceview.html

in conjunction with the Debug class:
http://developer.android.com/reference/android/os/Debug.html

sky said...

“牛b”in Chinese.

Rednoah said...

Hi Markus,

I tried to use Eclipse Memory Analyzer to analyze the memory leak issue described at http://android-developers.blogspot.com/2009/01/avoiding-memory-leaks.html,

but the report did not show the reference object to the "bitmap" which caused the leak.

Why? Is because of the Eclipse Memory Analyzer could not monitor the native memory?

Thanks.

Unknown said...

Hi rednoah,
If your talking about the "memory leak" report, almost all reports are based on heuristics e.g. they have no application/framework knowledge.
These heuristics only work as long as the leak is large. In your case the leak is probably not large.
One could build a new command to detect exactly this leak, but this command would have to know exactly how this works on android.

MAT my also not take the size of the bitmap into account, if this is a native object.

Regards,
Markus

Unknown said...

Nice article.
I have a question. I follow your steps to get a heap dump from my Android application. But, why didn't I see the usage of user-defined classes in the histogram view in Eclipse Memory Analyzer? It is possible to see histogram of user-defined classes?

Thanks.

Unknown said...

Hi all

i am done upto adb shell ps
after that if i am giving

adb shell kill -10

it is showing could not kill pid -10:no such process error.

can u pl tel me where i did mistake
even i am not getting .hprof in logcat

Unknown said...

The formatting of the commands was screwed up (seems to have been ok when I posted it).

I changed the post slightly.


You need to find the process id of your process using ps. then you use this pid in the kill command.

Unknown said...

Hi Markus Kohler

Thankq for your reply.

i am giving adb shell kill -10 pid

even after i am not getting .hprof file in logcat .am i missing anything else.can you please tell me where i did mistake again

Swifty said...
This comment has been removed by the author.
Swifty said...
This comment has been removed by the author.
Swifty said...

Some Good Advice Here many Thanks

Swifty

All Time Best Android Apps

Unknown said...

I am trying this on my actual device. When trying to kill -10 pid its giving this error
"could not kill pid 10308: Operation not permitted".

Unknown said...

Hi "Sugandh",
This only works if you have root permissions e.g. you need to run this command as root.

Regards,
Markus

Kirk A said...

Regarding Update 2, what IS the new command? The provided URL explains that 'kill' no longer works in v2.3, but does not suggest what DOES work. Even the ddms dump button tooltip reads "Dump HPROF file (not supported by this VM)".

Thank you for sharing the information here. Do you have recommendations for creating heap dumps in v2.3?

Kirk A said...

While ddms Dump HPROF wouldn't work for me in v2.3.1, it works now in v2.3.3. Again, thanks for the pointers.

Unknown said...

Thanks for the hint Kirk!
I'm pretty sure the page mentioned a new command. I have to research whether I was only dreaming ;-)
Haven't really tried 2.3, but will do so soon.

android app developer said...

Great information to be check the memory usage of the phone. I want to become a good android developer in the future. I have a good knowledge about the Core Java.

Android app development said...

This is one one of the genius post.By read your post we can get good idea about Internet marketing.This is one of the amazing post.Android app developers

Krishna R said...

Thrilled to use MAT on android/dalvik heaps!

I've been using Memory Analyzer before SAP donated it to Eclipse. Its bad that many developers use it ony for Memory leak analysis (and they prefer profilers at that). I've been evangelizing MAT to use for any kind of issues to look at the stateful objects in the heap.

It'd be great to expose a dynamic scripting (such as JS in jHat). I'm writing some groovy extensions for MAT to write sophisticated scripts (free-hand groovy/java or a DSL) to analyze complex object graphs and also expose Query & *Resolver extension-points.

Unknown said...

Hi Krishna R,
Thanks!
This sounds very cool?
Anything you could share?
Did you run into performance issues with scripting?

Scripting might be something I will investigate more deeply in the near future.

Regards,
Markus

Unknown said...

I meant "cool!" of course :-)

Facebook App Development said...

This site is really amazing and informative to me, i am impressed by this site.

Richard Byrdk said...
This comment has been removed by the author.
Top Movie Trailers said...

Amazing, the up to date information has enhanced my knowledge allot. Keep posting such useful posts so that I can reap maximum benefits from them. Thank you Quit Smoking Tips.

Rahul said...

Hi.. I am not getting below line while doing same steps as u instructed..

I/dalvikvm( ): threadid=7: reacting to signal 10 I/dalvikvm( ): SIGUSR1 forcing GC and HPROF dump I/dalvikvm( ): hprof: dumping VM heap to "/data/misc/heap-dump-tm- pid.hprof-hptemp". I/dalvikvm( ): hprof: dumping heap strings to "/data/misc/heap-dump-tm124026 3144-pid.hprof". I/dalvikvm( ): hprof: heap dump completed, temp file removed D/dalvikvm( ): GC freed 1292 objects / 92440 bytes in 11 sec D/dalvikvm( ): GC freed 215 objects / 9400 bytes in 963ms

I am getting not heap dump not complted their is no HROF..

Android Application Development said...

For android mobile memory anlyzer is very nice application. I use it when i take android mobile. information about to get that hep dump is really great and usefulf or all new users. Thanks for the great information

iPhone Game Development said...

Hey, I really glad to read this article. And definitely i will follow your steps for heap dumps. Thanks for the source.

Unknown said...

I havent any word to appreciate this post…..Really i am impressed from this post….the person who create this post it was a great human..thanks for shared this with us. Hire iPhone Developer

PHP Development India said...

Thanks for sharing this informative post with us. I am very happy to read your blog. It’s been very beneficial.

Wordpress Development India

Hire Java Developer said...

You are provided information very useful for android developer, and this app is really very user friendly; by the way thanks for share to all.

Hire Developer India

Mobile App Development India said...

I think about that Android Development is done entirely in Java, you are going to need a working understanding of the language and concepts. Otherwise, you will have a difficult time understanding and implementing Android concepts. Yup, Java has a lot of topics to learn. So does Android. You will need a good base in Java for the Android stuff to be meaningful.

Hire Android App Developer