The Blog

Guide to symbolicating iPhone app crash logs with Xcode 4.2

I was recently investigating one very strange crash in an app I’m working on. The app is distributed ad-hoc (through wonderful TestFlightapp.com) and each release is archived in Xcode 4.2. I got the .crash file from beta tester and dragged it into Xcode’s Organizer, which did symbolicate everything in the stack except two lines from my code.

Incident Identifier: 8B369A29-8421-4686-B1F6-9D66524937B5
CrashReporter Key:   746834fde1cb2033d4103df311e8f3b6ee9e8817
Hardware Model:      iPhone3,1
Process:         myApp [74]
Path:            /var/mobile/Applications/78EC36CC-DAF5-4910-9D60-A03DDC8E24BC/myApp.app/myApp
Identifier:      myApp
Version:         ??? (???)
Code Type:       ARM (Native)
Parent Process:  launchd [1]

Date/Time:       2011-10-27 11:37:55.841 +0100
OS Version:      iPhone OS 4.3.3 (8J2)
Report Version:  104

Exception Type:  EXC_CRASH (SIGABRT)
Exception Codes: 0x00000000, 0x00000000
Crashed Thread:  0

Thread 0 name:  Dispatch queue: com.apple.main-thread
Thread 0 Crashed:
0   libsystem_kernel.dylib        	0x3657aa1c 0x36569000 + 72220
1   libsystem_c.dylib             	0x366513b4 pthread_kill + 52
2   libsystem_c.dylib             	0x36649bf8 abort + 72
3   libstdc++.6.dylib             	0x36613a64 __gnu_cxx::__verbose_terminate_handler() + 376
4   libobjc.A.dylib               	0x3548806c _objc_terminate + 104
5   libstdc++.6.dylib             	0x36611e36 __cxxabiv1::__terminate(void (*)()) + 46
6   libstdc++.6.dylib             	0x36611e8a std::terminate() + 10
7   libstdc++.6.dylib             	0x36611f5a __cxa_throw + 78
8   libobjc.A.dylib               	0x35486c84 objc_exception_throw + 64
9   Foundation                    	0x3522d924 __NSThreadPerformPerform + 648
10  CoreFoundation                	0x31942a72 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 6
11  CoreFoundation                	0x31944758 __CFRunLoopDoSources0 + 376
12  CoreFoundation                	0x319454e4 __CFRunLoopRun + 224
13  CoreFoundation                	0x318d5ebc CFRunLoopRunSpecific + 224
14  CoreFoundation                	0x318d5dc4 CFRunLoopRunInMode + 52
15  GraphicsServices              	0x31254418 GSEventRunModal + 108
16  GraphicsServices              	0x312544c4 GSEventRun + 56
17  UIKit                         	0x319fbd62 -[UIApplication _run] + 398
18  UIKit                         	0x319f9800 UIApplicationMain + 664
19  myApp                      	0x00003a4c main (main.m:14)
20  myApp                      	0x00003a04 0x1000 + 10756
...
Binary Images:
    0x1000 -    0x67fff +myApp armv7  <a0b39fab741e34eb831048cc752d8e0c> /var/mobile/Applications/78EC36CC-DAF5-4910-9D60-A03DDC8E24BC/myApp.app/myApp

Great. The usual reason for this is that Xcode did not find .dSYM file. This is actually really strange since Xcode has the .xcarchive, saved in its default location. But no matter what I tried, it did not symbolicate properly. One possible reason that I’ve seen people mention is that Spotlight has not indexed the archives and you can force this with this Terminal command:

mdimport ~/Library/Developer/Xcode/Archives/

This did not help either. I tested Spotlight and strangely it could not find the app signature (from the bottom of the crash log excerpt above):

mdfind "com_apple_xcode_dsym_uuids == A06EC84E-53BF-3209-8B63-C0CAEBDB45B6"

At this point I tried to do it manually and in series of attemps encountered all kind of possible issues.

First, I had a custom symbolicatecrash script in /usr/local/bin/ from several months ago. This script often changes as Apple devs fix bugs in it. While they do that and ship the fixed version with next Xcode update, good souls on the internet have already resolved the given issue. Thus at some point I downloaded one of those custom versions and placed it there. Hence when I tried to run symbolicatecrash manually, it used that version. You can check this in Terminal, by typing which symbolicatecrash – if it finds the script anywhere in your default paths, it will show it. If not, then you are ok.

Second, in Xcode 4.2, the current version of this script is at this location: /Developer/Platforms/iPhoneOS.platform/Developer/Library/PrivateFrameworks/DTDeviceKit.framework/Versions/A/Resources/symbolicatecrash. Files in this private framework are not accessible from any other folder, so in order to run it manually you need to either use this full path or copy it to the folder where you acquired .crash log files are and run it from there.

Third, I extracted the .app and .dSYM file from the archive and placed in the same folder. I then tried to run the script from there, with all of these line (one by one):

./symbolicatecrash -v *.crash . > s.crash
./symbolicatecrash -A -v myApp_2011-10-27-113755_Alex-iPhone.crash myApp.app.dSYM > s.crash

None did anything differently. Things began to look really crazy at this point and I started doubting is this a crash report really from this version. Double-checked with tester, also double checked the .app.dSYM file as described here on Stack Overflow. All was fine, this was the binary used.

Forth, I tried to directly look for the given hex address in the crash log:

dwarfdump --lookup 0x00003a04 --arch armv6 myApp.app.dSYM

And this finally gave me what I needed:

----------------------------------------------------------------------
 File: myApp.app.dSYM/Contents/Resources/DWARF/myApp (armv6)
----------------------------------------------------------------------
Looking up address: 0x0000000000003a04 in .debug_info... found!

0x0000012c: Compile Unit: length = 0x00005a18  version = 0x0002  abbr_offset = 0x00000000  addr_size = 0x04  (next CU at 0x00005b48)

0x00000137: TAG_compile_unit [1] *
             AT_producer( "Apple clang version 3.0 (tags/Apple/clang-211.9) (based on LLVM 3.0svn)" )
             AT_language( DW_LANG_ObjC )
             AT_name( "/Users/aleck/dev/consulting/myApp/trunk/Classes/AppDelegate.m" )
             AT_entry_pc( 0x0000323c )
             AT_stmt_list( 0x00000112 )
             AT_comp_dir( "/Users/aleck/dev/consulting/myApp/trunk" )
             AT_APPLE_optimized( 0x01 )
             AT_APPLE_major_runtime_vers( 0x02 )

0x00000dc3:     TAG_subprogram [15] *
                 AT_sibling( {0x00000dfb} )
                 AT_name( "-[AppDelegate dealloc]" )
                 AT_decl_file( "/Users/aleck/dev/consulting/myApp/trunk/Classes/AppDelegate.m" )
                 AT_decl_line( 250 )
                 AT_prototyped( 0x01 )
                 AT_APPLE_isa( 0x01 )
                 AT_low_pc( 0x000039d4 )
                 AT_high_pc( 0x00003a6c )
                 AT_frame_base( r7 )
Line table dir : '/Users/aleck/dev/consulting/myApp/trunk/Classes'
Line table file: 'AppDelegate.m' line 254, column 5 with start address 0x0000000000003a02

Looking up address: 0x0000000000003a04 in .debug_frame... found!

0x00000100: FDE
        length: 0x0000000c
   CIE_pointer: 0x00000000
    start_addr: 0x000039d4 -[AppDelegate dealloc]
    range_size: 0x00000098 (end_addr = 0x00003a6c)
  Instructions: 0x000039d4: CFA=4294967295+4294967295

Or so I thought. When I looked into my code at this line, this was in dealloc method, where NSPersistentStoreCoordinator was released. And that was really nutty place to have a bug. At this point I was completely stumped and had no idea what else to try.

Luckily in this case, tester was able to give login credentials so I could repeat the scenario using his data, while debugging the app on the device. And found that I had a badly formatted NSPredicate which worked in 99.9% of cases and this tester stumbled on 0.1% where it did not. Great find, it would be a serious head-scratcher if app went live with this.

But I still don’t understand why Xcode 4.2 won’t see the .dSYM file and while the script itself won’t symbolicate this .crash log. Honestly, I can’t help but think this is seriously more complicated than it needs be. But it is as it is. Here some other useful stuff I encountered while working on this.

Place where Xcode puts .crash logs picked from your testing devices: ~/Library/Logs/CrashReporter/MobileDevice/<your device name>.symbolicated

Various possibly helpful links:
· http://www.goosoftware.co.uk/blog/the-symbolicator-helps-those-who-help-themselves/
· https://devforums.apple.com/message/404524
· http://stackoverflow.com/questions/2697067/symbolicate-adhoc-iphone-app-crashes

If you have any ideas that could help, please write in comments.

The thin fun line

Tweetbot - IMG_1037

In the last few days, there’s been quite a storm in the Twitter tea pot, regarding the new hit iOS client, Tweetbot. All Tapbots apps have completely custom UI, sounds and interaction but they always managed to make it so and still keep great performance. This is the main reason for their success – the fun part in using them did not come with a price (like UI lag).
However, Tweetbot faces quite a bit criticism that its custom UI hinders the UX of the app.

Many things have been said regarding this but I find Ben Brooks’ series of complaint posts particularly misplaced. He argues that Tweetbot’s custom UI over-compliates the things without bringing anything in. He’s wrong, especially this:

Gestures in the case of Tweetbot aren’t adding anything to the all important UX — in fact I would think they are detracting from it by straying so far from conventional iOS norms.”

Tweetbot isn’t forcing you to learn anything new. You can use the entire app with one and only gesture every single iPhone user knows – simple short tap. Tap to reveal most-used actions over a tweet. Tap the view button to load the tweet in full-screen view and reveal additional options (conversation, related tweets, block/spam report etc).
What I would have liked is that Mark Jardine has not used an eye for this icon, but instead used the default disclosure indicator, like this:

A better view icon

That button moves the view hierarchy further along, so the default system icon should have been used. Apart from this small hickup, the UI/UX in Tweetbot great – lacks nothing, hinders nothing else.

Seriously, just try – you don’t have to know any of the gestures (swipes, multiple taps or long taps) to use the app. But when you do discover them, you find yourself using the app that much faster.

  1. User details are one tap on avatar away, important actions over user are one long tap away
  2. Actions over tweet are one tap away
  3. Share options are another tap away after the previous one (instapaper, email etc)
  4. Conversation is one swipe away. Related tweets (answers to the tweet) are one swipe away
  5. Reply is triple-tap away (in my case)

etc. I think it’s clear – the UI is easy to use, learning curve is almost straight line (apart from that eye icon). But the UX is greatly enhanced with gestures and it brings you that wonderful feeling of being proud that you are the power user.

You should also read this feature showdown which, in its second part, shows why Tweetbot is so good. It’s great for reading the timeline and answering here and there. I can’t imagine anyone (apart from the most egoistical jerks) writing more tweets than reading them, so this is the advantage to have. The feature where Tweetbot is able to properly load the part of the timeline you may have missed during the night is invaluable to me. Official Twitter client often loads this and then loses my place where I was so I have to scroll and find the last tweet I was on.

OPPO’s fantastic support service

oppo-981hd

I have OPPO DV-981HD player for several years now. It’s a great piece of consumer electronics – from the moment I put it on it worked great. No issues, no fuss, it worked exactly as advertised. Until few weeks ago when it died. In the middle of movie, it saved a bookmark and shut down. No reactions to buttons, it just seemed dead.

The player was long ago out of warranty. Plus, I brought it to Serbia from UK, one of the last pieces the importer had. OPPO does have a “send it to us for repair” service, but the cost of sending to USA from Serbia would be more than the player is now worth. Plus, experience with other companies tells me that prices for obsolete (OPPO does not sell this model anymore) and out-of-warranty parts are outrageous.

So I emailed OPPO support, simply hoping for an advice what could be wrong so I can try my luck with local repair shops. I was hoping it’s simply some part of the power board. That was on Saturday afternoon (CET time zone).

This is where it becomes awesome. First, I got answer in less than 24h. Second, the answer said that OPPO can help me by sending me a replacement power board and a front-end display. Third, the price for the parts is mere $49. And then to top it off, they responded to my follow-up questions on Sunday and Monday (which was a bank holiday in US). I paid and they shipped the parts the very next day; the package reached me by the end of the week (sent by USPS).

Amazing, amazing support service, way better than I expected. I replaced the parts and the player is working as good as ever – silently and awesomely.

If you are in need of an excellent Blu-ray player, do not even think about anything else – buy one of the OPPO players they have on offer. I’m certain they are amazing just as this DVD player is. I mean, just look at the feature set, the customer testimonials and rave reviews they consistently get for their products.

We need more companies like OPPO. Kudos, masters.

Fighting feature creep

Giant-Swiss-Army-Knife-8x6

Feature creep is an easy trap to fall into. As developer and software vendor, you always want to please your users. So when a request comes in and it’s eloquently and nicely laid out, it’s hard to resist. I experienced it so much while working on Run Mate 1.1, which is the main reason why it took me 3 months to publish it.

Never again. I hope.

I have a new weapon against it now — it’s a wonderful thought that Wil Shipley said in an interview for Mac Developer Network podcast:

…Technology is just being too complicated. People just don’t enjoy using it, they don’t get it, they’re not getting the most out of it, they’re not able to use the feature they have and if we radically simplify it, then people suddenly get a lot more out of it.
They actually use more features if you give them less features.

MDN podcast is wonderful resource. I had a large backlog of various podcasts — this Shipley interview is in MDN Show 003 from back in July 2009 — which I have recently cleared out. I never check what is in any of the shows, I like when they surprise me. This interview with Wil is chock full of great thoughts.

Another fantastic feature of the MDN Show is World according to Gemmell where Matt Gemmell picks an UX subject and really, really hummers it down.

Note: Steve Scott, the man who ran MDN, has retired that show and now has a new show called iDeveloper Live. It’s more dynamic than MDN Show (more people) plus it’s recorded live and you can be part of it through the chat.

Debugging [CALayer retain]: message sent to deallocated instance

While working recently on an iPhone app, I had a subclass of UITableViewCell with a specific indicator, all defined like this:

OBJC:
  1. @interface EmailListCell : UITableViewCell {
  2. UIImageView *indicator;
  3. }
  4. @property (nonatomic, retain) UIImageView *indicator;

In the implementation part — since the property is retained, I automatically added this:

OBJC:
  1. @implementation EmailListCell
  2. @synthesize indicator;
  3.  
  4. - (void)dealloc {
  5. [indicator release];
  6. indicator = nil;
  7.  
  8. [super dealloc];
  9. }

This is what I usually do, mostly mechanically, to not forget to add proper releasing. And it came back to bite me this time.

It was because of the initializing code I wrote after the code above:

OBJC:
  1. - (id)initWithStyle:(UITableViewCellStyle)style reuseIdentifier:(NSString *)reuseIdentifier {
  2. if (self = [super initWithStyle:style reuseIdentifier:reuseIdentifier]) {
  3.  
  4. indicator = [[UIImageView alloc] initWithImage:widgetEmpty];
  5. [self addSubview:indicator];
  6. [indicator release];
  7.  
  8. }
  9. return self;
  10. }

When testing this table view, it would — in some cases only — crash the app, with one of these messages shown:

*** -[CALayer retain]: message sent to deallocated instance 0x5c73490

modifying layer that is being finalized 0x713a170

These are thrown when you attempt to access UIView of any sort that is in the process of being deallocated. Catching these things is very tricky, because the objects mentioned are already gone, so even if you setup breakpoint to objc_exception_throw, it won’t help you much.

So, do you see where the issue is? :)

I am releasing indicator view twice. First time when it is created and second time in dealloc. The latter is not needed, since all subviews of the cell’s view will automatically be released along with it. The code I wrote would be correct if the init line used the property:

OBJC:
  1. self.indicator = [[UIImageView alloc] initWithImage:widgetEmpty];

By using self, I’m actually raising the retain count to 2, so two release calls would be fine. Thus, the proper initialization code would be:

OBJC:
  1. UIImageView *ind = [[UIImageView alloc] initWithImage:widgetEmpty];
  2. self.indicator = ind;
  3. [self addSubview:ind];
  4. [ind release];

And then the indicator release in the dealloc is mandatory (and also a proper way when dealing with retained properties).