The Boot ROM code will detect the boot media using a system register that maps to some physical ROM on the asic. This is to determine where to find the first stage of the boot loader. Once the boot media sequence is established the boot ROM will try to load the first stage boot loader to internal RAM. Once the boot loader is in place the boot ROM code will perform a jump and execution continues in the boot loader.

The boot loader is a special program, separate from the Linux kernel that is used to set up initial memories and load the kernel to RAM. On desktop systems the boot loaders are programs like GRUB and in embedded Linux uBoot is often the boot loader of choice. Device manufacturers often use their own proprietary boot loaders. The requirements on a boot loader for Linux running on an ARM system can be found in the Booting document under /Documentation/arm in the kernel source tree.

The first boot loader stage will detect and set up external RAM. Once external RAM is available and the system is ready to run something more significant, the first stage will load the main boot loader and place it in external RAM. The second stage of the boot loader is the first major program that will run. This may contain code to set up file systems, additional memory, network support and other things. On a mobile phone it may also be responsible for loading code for the modem CPU and setting up low level memory protections and security options.

Once the boot loader is done with any special tasks it will look for a Linux kernel to boot. It will load this from the boot media (or some other source depending on system configuration) and place it in the RAM. It will also place some boot parameters in memory for the kernel to read when it starts up. Once the boot loader is done it will perform a jump to the Linux kernel, usually some decompression routine, and the kernel assumes system responsibility.

So the outline of the Android system boot is as follow…

1. On power-up, CPU is uninitialized – wait for stable power
2. Execute Boot ROM (hardwired into CPU)
3. Locate the first-stage boot loader
4. Load the first-stage boot loader into internal RAM
5. Jump to first-stage boot loader’s memory location to execute it
6. First-stage boot loader runs
7. Detect and initialize external RAM
8. Locate the second-stage boot loader
9. Load the second-stage boot loader into external RAM
10. Jump to the second-stage boot loader’s memory location to execute it
11. Second-stage boot loader runs
12. Setup file systems (typically on Flash media)

Steps 1 to 11 are pretty much standard for any kind of linux based system. For the sake of speaking step 12 may also be standard for Flash based devices. Since Android based devices also comes under that category so lets start understanding these steps in greater details starting with File system. I shall outline the rest of the steps of system start up whenever i post the article related those.

What is File system?
I think it is better to quote wiki here rater than to invent words for the well know concept. Here it is…

“A file system (filesystem) is a means to organize data expected to be retained after a program terminates by providing procedures to store, retrieve and update data, as well as manage the available space on the device(s) which contain it. A file system organizes data in an efficient manner and is tuned to the specific characteristics of the device. There is usually a tight coupling between the OS and the file system. Some filesystems provide mechanisms to control access to the data and metadata. Ensuring reliability is a major responsibility of a filesystem. Some filesystems provide a means for multiple programs to update data in the same file at nearly the same time.”

Each storage device architecture requires a file system that understand that hardware and uses its features in an optimal manner. We have multiple variation of storage devices. In the context of Android what we are talking is Flash based storage. Below is the description of flash based storage.

Flash memory is a non-volatile computer storage chip that can be electrically erased and reprogrammed. It was developed from EEPROM (electrically erasable programmable read-only memory) and must be erased in fairly large blocks before these can be rewritten with new data. The high density NAND type must also be programmed and read in (smaller) blocks, or pages, while the NOR type allows a single machine word (byte) to be written and/or read independently.

The NAND type is primarily used in memory cards, USB flash drives, solid-state drives, and similar products, for general storage and transfer of data. The NOR type, which allows true random access and therefore direct code execution, is used as a replacement for the older EPROM and as an alternative to certain kinds of ROM applications. However, NOR flash memory may emulate ROM primarily at the machine code level; many digital designs need ROM (or PLA) structures for other uses, often at significantly higher speeds than (economical) flash memory may achieve.

NAND or NOR flash memory is also often used to store configuration data in numerous digital products, a task previously made possible by EEPROMs or battery-powered static RAM.

Flash based file system: YAFFS2
YAFFS is abbreviation for Yet Another Flash FileSystem 2. This is a special purpose file system which works best for Flash bases storage devices in terms of performance and longevity-wise. Since the YAFFS is designed to run directly or separated by only a very thin layer of abstraction for example mtdblock on flash chips. They take care of wear-leveling and use flash-friendly data structure. Essentially the device on its own is not required to have any sort of wear-leveling.

Future of Android Flash based storage filesystem: EXT4
EXT4 is designed for concurrent access by multi-core platform. As we have noticed that recently Android is being delivered on platform with dual-core and quad-core platform, the choice of moving to EXT4 seems to fulfill this goal. Google has announced that with Android 2.3 which is also known as Gingerbread and lead product as Samsung Nexus S would use EXT4. Other vendor may follow the trend.

There are many SD-Cards that take care of all these issues since they have h/w assisted wear-leveling. It is also observed that these sort of SD-Cards coupled with normal off-the-self filesystem deliver better performance and better reliability than what the special purpose like YAFFS could.

Implementation
Clearly any storage system which does not have h/w wear-leveling can not or should not be moved to EXT4 since it would result in constant wear out of the device’s storage and eventual death. What could be the quick and cheapest method to move to EXT4 filesystem for vendors.

The solution is to introduce a NAND flash chip that has the MMC interface. To the processor this NAND chip would appear to be a SD-CARD and hence EXT4 can be applied. Along with that MMC interface the storage device must provided wear-leveling either at s/w or h/w level to compensate for EXT4 lack of wear-leveling.

Is EXT4 any concern to me?
An increasing proportion of devices, notably including the Nexus S, are going to be moving from YAFFS to theext4 filesystem, which buffers much more aggressively; thus you need to be more assertive about making sure your data gets to permanent storage when you want it to.

If you just use SharedPreferences or SQLite, you can relax, because we’ve made sure they Do The Right Thing about buffering. But if you have your own on-disk format, keep in mind that your data doesn’t actually consistently reach the flash chip when you write() it or even when youclose() it. There are several layers of buffering between you and the hardware! And because of ext4 buffering policy, any POSIX guarantees that you thought you had before (but actually didn’t), you especially don’t have now.

Below is quick reference to wiki on the problem outlined above..

Delayed allocation and potential data loss
Because delayed allocation changes the behavior that programmers have been relying on with ext3, the feature poses some additional risk of data loss in cases where the system crashes or loses power before all of the data has been written to disk. Due to this, ext4 in kernel versions 2.6.30 and later automatically detects these cases and reverts to the old behavior.The typical scenario in which this might occur is a program replacing the contents of a file without forcing a write to the disk with fsync. There are two common ways of replacing the contents of a file on Unix systems:[13]

fd=open(“file”, O_TRUNC); write(fd, data); close(fd);

In this case, an existing file is truncated at the time of open (due to O_TRUNC flag), then new data is written out. Since the write can take some time, there is an opportunity of losing contents even with ext3, but usually very small. However, because ext4 can delay allocating file data for a long time, this opportunity is much greater.
There are several problems with this approach:

1. If the write does not succeed (which may be due to error conditions in the writing program, or due to external conditions such as a full disk), then both the original version andthe new version of the file will be lost, and the file may be corrupted because only a part of it has been written.
2. If other processes access the file while it is being written, they see a corrupted version.
3. If other processes have the file open and do not expect its contents to change, those processes may crash. One notable example is a shared library file which is mapped into running programs.

Because of these issues, often the following idiom is preferred over the above one:

fd=open(“file.new”); write(fd, data); close(fd); rename(“file.new”, “file”);

A new temporary file (“file.new”) is created, which initially contains the new contents. Then the new file is renamed over the old one. Replacing files by the “rename” call is guaranteed to be atomic by POSIX standards – i.e. either the old file remains, or it’s overwritten with the new one. Because the ext3 default “ordered” journalling mode guarantees file data is written out on disk before metadata, this technique guarantees that either the old or the new file contents will persist on disk. ext4′s delayed allocation breaks this expectation, because the file write can be delayed for a long time, and the rename is usually carried out before new file contents reach the disk.

Using fsync more often to reduce the risk for ext4 could lead to performance penalties on ext3 filesystems mounted with the data=ordered flag (the default on most Linux distributions). Given that both file systems will be in use for some time, this complicates matters for end-user application developers. In response, ext4 in Linux kernels 2.6.30 and newer detect the occurrence of these common cases and force the files to be allocated immediately. For a small cost in performance, this provides semantics similar to ext3 ordered mode and increases the chance that either version of the file will survive the crash. This new behavior is enabled by default, but can be disabled with the “noauto_da_alloc” mount option.[13]

The new patches have become part of the mainline kernel 2.6.30, but various distributions chose to backport them to 2.6.28 or 2.6.29. For instance Ubuntu made them part of the 2.6.28 kernel in version 9.04 (“Jaunty Jackalope”).[14]
These patches don’t completely prevent potential data loss or help at all with new files. No other filesystem is perfect in terms of data loss either, although the probability of data loss is lower on ext3. The only way to be safe is to write and use software that does fsync when it needs to. Performance problems can be minimized by limiting crucial disk writes that need fsync to occur less frequently.[15]

So take home from this is that think twice before you pull out the battery of your phone. You may leave your phone in non-usable state and recovery may be only option left.

References:-

1. http://en.wikipedia.org/wiki/Ext4
2. http://en.wikipedia.org/wiki/Flash_memory
3. http://android-developers.blogspot.com/2010/12/saving-data-safely.html

Let me know your comments on this article. leave your feedback and questions in the comment section. I would be glad to help you out and would appreciate if you found some corner issues with above understanding. Peer review rocks!

Stay tuned for more articles to come.

Till then be safe and keep coming back.

Few days back i attended Google DevFest in Hyderabad and that was the biggest crowed puller. See it for yourself. It makes the point very clear that if you post on the recent hot activities then definitely you are going to get some traction out of it.

So the lesson from this exercise was that i should post more and often on technical stuff.

Wooo!! Hurrrrraaaaaaay! That’s my forte…..!

Hmmm! what technical stuff i should muse about? What could be better than the Android Internals.

For starters i am not talking about Android SDK API’s but to the core which makes those SDK APIs work the way they work.
I hope my upcoming articles would be useful for Platform developers and hobbits who are trying to either port or integrate the Android OS on their hardware.

Till then be safe and keep coming back.

I reached the venue by 8:00 AM !! Those who are familiar with me knows that a lazy bum like me has never made it to my own office by that time. OK..OK been there and did it once, when a nearby transformer blown out and stole my sleep, but you know that was just an awful experience.

It was HICC(Hyderabad International Convention Center) near to Novotel Hotel. I have been to that place once before too. Had Apple interview…more on that later, may be in some other post.

It was Google’s 2nd developer fest in Hyderabad. Previously they did it in Feb 2010. I was really excited and it was quiet visible from my outlook. Dressed properly as a developer, untidy hair, week long unsaved face with sleep deprived eyes and slight mischievous in the corner of eyeballs …you know what i mean. Sleeves folded and back pack with IBM T410 customized with Apple logo on the LCD lid.

They started the event by 9:30AM. Got a chance to meet few more folks from company and had wonderful chat with them. Then we rushed to occupy the front row. I was surprised that front row tables were unoccupied. How come on earth with an event like these and people are not holding front rows. Perplexed, but later i got it all answered. Majority of attendees were students. And you know that student and back-benches during lectures are synonyms in India. To some it was day long lecture only

Anyway, we started. The echo was really bad. It was ruining the whole experience but who cares when you are getting such a good tech-talk. They explained many stuff which they just launched a day before like Native Client Implementation in Chrome and PlayN.

Native Client support in Chrome. Man this is a big thing…if you know what i am talking about. For example, you have a game written in native language which runs directly on x86 hardware without any middle layer(like that of Java byte code interpretation). Now with this Chrome implementation you would be able to run it directly from browser. How cool that would be!! Of course Google has to make sure security model of browser is not broken else it would be just like the same hell as created by Windows.

Heard that they announced $2^13 for the 1st price winner who could break the system. Check it out here. http://code.google.com/contests/nativeclient-security/

I attended mostly Android, Games, HTML5 and Maps related talks and skipped Clound and Google Apps sections. Later during afternoon had the awesome lunch. Really enjoyed that one.

Another memorable experience was the talk with new startup companies like “Zomato” and “TelipoortMe”. They talked about there experience and influence of Android in getting them to where they are. I personally liked the concept and ISP of Zomato business model. It was nothing new but they were really executing it well. Just like “Flipkart.com”. In today’s time, execution matters more than the actual idea. Well done guys!

Got very tired by the closing time. Rushed to home and slept till midnight.

Overall, it was one of the one of best days of my life!!

See you later.

Have you noticed a drug addict. As long as he is under influence of it, he is feeling great but when the clouds clears, he feels a void. Deprived of a meaning of life. This is called withdrawal symptom.

I have started feeling the same void each and every time, i finish something. Like up and down of some hill-ride, you feel good when you are on height but next moment you are going down and then looking for next height. What if the next height is not quiet evident from your current stand, you roam around to find something, it sucks!

Yes, it sucks. It sucks when i am looking what next to do. It sucks at the feeling of withdrawn. It sucks when i am not productive and still have to be in office for atleast 8 hrs.

Sometimes i think, it would be a better world if we had a rule to leave office if your productivity for the day drop below 100%

Think about it.
See you later.

Over the past few years, scientist, journalist and blogger are discovering a new species inhabitant on our dear earth. Only small information has been leaked on this new kind of species which is being code-named as apple-fanboy!! This specie which is gender agnostic and found to be infectious. Several boys, girls, aged man, women, toddler and even as few animals (cats) are told to be infected by the symptom know to be specific to apple-fanboy.

The editor of this post did an extensive study on the said subject and lately he also started exhibiting these symptoms. We have taken care and locked him in room with no Window(7) or iMac Door. All the mac-mini escape route has been sealed and the editor is on time-capsule diet till further advised by his physician.

It was essential to mention his deteriorating health condition so that you can appreciate the hard work and kind of efforts he has put to jolt down this research. We pray he gets well soon.

Now without any further ado, we proceed towards the characteristics of apple-fanboy.

1. From outside they appear calm and sober but inside they have an inherent feeling of being afloat or being superior to other non-apple-fanboy.

2. They tend not to make any mistake in showing their infectious accessories. Better knows as ipod, ipad, imac etc.

3. These species are known to get hyperactive around June-sep every year. The reason is little known but it is observed that a large amount of fanboys gather in a big conference room on some specific day. It is also discovered that from that day onward their desire to get more infection gets increased by manyfolds

4. Around that same time, more and more innocent Human and animals too get that infection by watching youTube or TV broadcast of the proceedings of that conference.

5. Editor also observed that in some cities around the world some fanboy gets crowded around some stores called apple store from the midnight. some even do the camping in one week advance.

6. To fight this little known virus mayhem a new kind of self defense species was brought to life by mother nature. From the google search, this little defense robot is known as Droid.

7. Droid quickly mutated or got scattered by the mighty apple-fanboy clash…. its fragmentation exact cause is not known as of now. What we know is that there are now many droid versions around the world and they are working very hard to eliminate these sort of immortal species. The bad news is that they are facing a hard battle with no sign of victory in sight.

8. It is also observed that the infected person/animal is happily willing to get more infected with newly mutated version and this desire is found to be more in the oldest infected species than newly infected.

9. Recently we happened to note that this ever growing infection is more related to an enigmatic, larger than life and tall figure know as Steve Jobs.

10. A recent underground announcement of getting infected via preorder on internet leads to such a chaos that resulted in internet breakdown at several apple stores and ATT caves.

11. We thanks ATT to keep stronger hold on this virus and keeping it in its own caves. They did not allow this virus to get spread to other caves. if the virus could have spread then only god knows we would not be watching the Resident Evil rather we would be living it.

We went through all the scrap notes of the editor and found that he has jolted down few points on how to prevent this virus from spreading or getting infected.

1. Do not eat apples, do not be in company of a person having first name, last name of middle name that sounds like Steve or Jobs. If you can not avoid that person better leave the company or country for good. Its your life which is more important.

2. Stay away from people or teens, toddler or cats who sounds or smells like infected with this virus. DO NOT EVER try to look, touch or feel their accessories. Remember if you even glanced over or touched these devices you will get infected for life. THERE IS NO CURE.

3. Avoid the streets where there is apple stores. Avoid infinite loop, C. at all cost. If you can not avoid it then always keep looking opposite the apple store walls. If you happened to get a glance of the store inside immediately hit yourself with brick or start omitting. The quickly you get out of that lane the better.

4. Turn off your TV, switched of your cell from june to september. Cut all the wires related to internet. Better go to Himalayas.

5. Burn the news paper to ashes if it contains names like ipod, iphone, ipad or Steve Jobs.

6. Do not try to pursue apple-fanboys against apple. It is futile and dangerous. They tends to get ignited, provoked and highly reactive. Better stay low and fled away from the spot if you start seeing these signs in a apple-fanboy.

Regards,
Yash.

ps: This post is only for humor. The names of people mentioned here are NOT meant for any reference or illustration. I apologies if my sense of humor hurts.

If wave is not visible follow it here..

https://wave.google.com/wave/waveref/googlewave.com/w+CgDCfhivD

[wave id="googlewave.com!w+CgDCfhivD"]

Kasab convicted. May be announced capital punishment. OK. What else…. What nothing!!! What the hell?? What about the handlers in pakistan? What about the associates in India? What about the Headley(who knows he already got the witness protection??).

Kasab is charged with “Waging war on India”. So was it Kasab personal motive to “wage war on India”. He is just a pawn and by so much media coverage and his widely talk about trial, i am sure , pakistan will have a twisted story to tell in Pak villages. My hanging Kasab we are going to create thousands more Kasab.

Indian government should focus on surgical strike to eliminate LeT and Pak supported terrorist cells instead of cutting the leaves, like Kasab. He is culprit and must be punished but we must act on the actual reason for his motive. The backbone, which is capable of producing more Kasab than our judicial department can hang.

Looks like our politician are too busy doing scams that they don’t have time to think on the real issues for which they are elected.

Wake up and find offensive or diplomatic solution to the Pak operated terrorism. You lazy, deaf bastards, do you need another Bhagat Singh to Open Your Ears!!!

Is not 26/11 enough of proof that it was Pak initiated provocation of war. So that Pakistan can justify the inability to fight Afgan and can still vouch to US to get more Money and ammunition that can be posted in indian LOC??

Had it been US soil on which massacre happened, pak would have been missing from the world map by this time.

Excuse me for offensive post but could not control the emotions. Mark my words, once Kasab is gone, there would not be any followup. Neither by government nor by Media on 26/11. It would get recorded in history books and candle would be lit on 26/11 eve.

Is that how we want to remember 26/11????