Learnt from bugs

1049195

• gl uses kgsl through kgsl_ioctl()
• gralloc uses ion memory

1054011

•  HTMLIFrameElement::SetAttr()
     nsElementFrameLoaderOwner::LoadSrc()
       nsFrameLoader::LoadFrame()
         nsFrameLoader::LoadURI()
           nsDocument::InitializeFrameLoader()
             nsDocument::MaybeInitializeFinalizeFrameLoaders()
                nsFrameLoader::ReallyStartLoading()
                  nsFrameLoader::ReallyStartLoadingInternal()
                    if (mRemoteFrame) {
                      TabParent::LoadURL()
                        PBrowserParent::SendLoadURL()
                        --- IPC ---
                        TabChild::RecvLoadURL()
                          nsIWebNavigation::LoadURI()
                            nsDocShell::LoadURI()
                    } else {
                      nsDocShell::LoadURI()
                    }

1045715

•  Console::Notify()
     Console::ProcessCallData()
       NotifyObservers("console-api-log-event"...)
         CAL_Observe()
           WCA_onConsoleAPICall()
             WCA_preapareConsoleMessageForRemote()
             DSC_send()

1014649

• Escape analysis
  

  When a variable (or an object) is allocated in a subroutine, a pointer to the variable can escape to other threads of execution, or to calling subroutines. Escape analysis determines all the places where a pointer can be stored and whether the lifetime of the pointer can be proven to be restricted only to the current procedure and/or thread.

• Tail call
  

  Tail calls can be implemented without adding a new stack frame to the call stack. Most of the frame of the current procedure is not needed any more, and it can be replaced by the frame of the tail call, modified as appropriate (similar to overlay for processes, but for function calls).

1050122

•  window.parent
   parent@dom/interfaces/base/nsIDOMWindow.idl
   get_parent()@objdir-gecko/dom/bindings/WindowBinding.cpp
     nsGlobalWindow::GetParent()
• Mochitest
• runner_cls.start() [testing/mochitest/runtests.py]
    ProcessHandlerMixin.run() [testing/mozbase/mozprocess/mozprocess/processhandler.py]
      self.processOutput()
        self.outThread.start()
          while lines (stdout) != "": self.processOutputLine()
  runner.process_handler.wait()
    while self.outThread.isAlive(): self.outThread.join()
• SimpleTest.finish() [testing/mochitest/tests/SimpleTest/SimpleTest.js]
    parentRunner.testFinished()
      TestRunner.testFinished() [testing/mochitest/tests/SimpleTest/TestRunner.js]
        runNextTest()
          TestRunner.testUnloaded()
            TestRuner.runNextTest()
              TestRunner.onComplete()

1064800

• MTBF
  
  • test_mtbf_fmradio_turn_on_off.py (MtbfTestCase.setUp > GaiaTestCase.setUp)
      MTBF_FmRadio.launch()
        Base.lauch() [gaia-ui-tests/gaiatest/apps/base.py]
          GaiaApps.launch() [gaiatest/gaia-test.py]
            self.marionette.execute_async_script("GaiaApps.lauchWithName")
              self._send_message('executeAsyncScript') [marionette.py]
     --- marionette_server.js ---
     --- marionette_listener.js ---
     executeAsyncScript()
       executeWithCallback()
         createExecuteContentSandbox()
           Cu.Sandbox()
             new Marionette()
         Cu.evalInSandbox()
  • $ adb -s 3739ce32 forward tcp:2829 tcp:2828
    $ ANDROID_SERIAL=3739ce32 MTBF_CONF=conf/flame_v210.json MTBF_TIME=1d python mtbf.py --testvars=testvars.json --address=localhost:2829 tests/test_dummy_case.py

1094010

• TabChild::RecvLoadURL()
  ...
    nsJARChannel::AsyncOpen(jar:remoteopenfile:///...application.zip!/index.html)
      nsJARChannel::LookupFile()
        RemoteOpenFileChild::AsyncRemoteFileOpen()
  ...
  TabChild::RecvCacheFileDescriptor()
    nsJARChannel::OnRemoteFileOpenComplete()
      nsJARChannell::OpenLocalFile()
        nsJARChannel::CreateJarInput()
          nsJARChannel::OpenLocalFile()
            nsInputStreamPump::AsyncRead()
              nsIStreamTransportService::CreateInputTransport()
              nsInputStreamTransport::OpenInputStream
                NS_AsyncCopy()
                  ...
                    nsAStreamCopier::Start()
                      nsAStreamCopier::PostContinuationEvent()
                        nsAStreamCopier::PostContinuationEvent_Locked()
                          nsIStreamTransportService::Dispatch()
                          ---
                          nsJARInputStream::Read()
                            nsJARInputStream::ContinueInflate()
                          NS_NewInputStreamReadyEvent()
                          nsInputStreamReadyEvent::OnInputStreamReady()
             nsInputStreamPump::EnsureWaiting()
                nsPipeInputStream::AsyncWait()
                  NS_NewInputStreamReadyEvent()
  ...
  nsInputStreamReadyEvent::Run()
    nsInputStreamPump::OnInputStreamReady()
      nsInputStreamPump::OnStateStart()
        nsJARChannel::OnStateStart()
          nsDocumentOpenInfo::OnStartRequest()
  ...
  nsInputStreamReadyEvent::Run()
    nsInputStreamPump::OnInputStreamReady()
      nsInputStreamPump::OnStateTransfer()
        nsJARChannel::OnDataAvailable()
          nsDocumentOpenInfo::OnDataAvailable()
      ...
  ...
  nsHtml5LoadFlusher::Run()
    ...
    nsJARChannel::AsyncOpen(...css)
      nsJARChannel::OpenLoaclFile()
        nsInputStreamPump::AsyncRead()
          nsInputStreamPump::EnsureWaiting()
            EnsureWaiting()
              nsPipeInputStream::AsyncWait()
                NS_NewInputStreamReadyEvent()

1068232

• IDBCursor::Continue()
    BackgroundCursorChild::SendContinueInternal() > SendContinue()
  --- IPC ---
  Cursor::RecvContinue()
    TransactionThreadPool::TransactionQueue::Dispatch()
  --- Thread Switching ---
  ContinueOp::DoDatabaseWork()
  --- Thread Switching ---
  ContinueOp::SendSuccessResult()
    Cursor::SendResponseInternal() > SendResponse()
  --- IPC ---
  BackgroundCursorChild::RecvResponse()

1110624

• HTMLIFrameElement.getCanGoForward()
    BrowserElementParent.getCanGoForward()
      sendAsyncMessage('browser-element-api:call', { msg_name='get-can-go-forward', ...})
      --- IPC ---
      BrowserElementChildPreload._recvCanGoForward
        sendAsyncMessage('got-can-go-forward', { successRv: docshell.QueryInterface(Ci.nsIWebNavigation).canGoForward })
  
• TabChild::Init()
    do_CreateInstance(NS_WEBBROWSER)
      nsWebBrowser::Create()
        SetDocShell(do_CreateInstance(docshell))
    mWebNav = do_QueryInterface(WebBrowser)  // Will be used in TabChild::RecvLoadURL()

Optimize SpiderMonkey JSOP_IN

This is what I am currently working on, see bug 1046792.

Just had a meeting with Kannna last night, some notes (not sure whether I misunderstood, will double check later):

Telescoping shape optimization

If there's a chain A->B->C->D, and both A and D have property |x|. When |x| is added to B or C, D's shape will be updated. So when you know A and D have property |x|, you can check only A and D's shape to make sure the property is not updated on the chain next time.

Shadow property

For instance there's a prototype chain A->B, if both A and B have a property |x|, then B's |x| is in the shadow of A's |x|, which can't be seen.

Baseline stubs

- Can be shared among different ICs, see pointer |stubCode_| of class ICStub.
- The key of ICStubCompiler is the key of a hash map storing stub codes.
- It is removed rarely after attaching, for example when fallback notices too many are created.

ValueOperand

R0 actually holds a boxed value, which occupies 2 registers on 32bit platform, for example r3, r2 on ARM (see src/jit/arm/BaselineRegisters-arm.h). One is for type and one is for payload.

Nuwa

There's a process in Firefox OS similar to Android's Zygote, which uses fork() to spawn application processes and let them share memory pages by copy-on-write. It helps on lowering memory requirement and faster application launching. It is named Nuwa [1], a goddess in ancient Chinese mythology.

When look at it closer, you will find out things are not as easy as simply calling fork(). For instance, the threads created before fork() in Nuwa process are not inherited in child process; the open file descriptors for epoll from Nuwa process can't be used by child processes for IPC. We need to deal with those issues before we can earn some memory.

I'll try to explain how threads are recreated in child process.

First of all, we use the wrap feature [2] of ld to wrap some pthread functions. For example, all calls to pthread_create() will call the function __wrap_pthread_create() defined in Nuwa.cpp [3] instead.

When pthread_create() is called in Nuwa process, __wrap_pthread_create() new a thread info structure to store related data and keeps it in a linked list sAllThreads. Also a stack is allocated for the pthread being created. The actual start routine is then called in _thread_create_startup().

Once the thread runs and calls any of epoll_wait(), poll(), pthread_mutex_lock(), pthread_cond_wait(), and pthread_cond_timedwait(), it will setjmp() and call pthread_mutex_lock() on an already locked mutex to block the thread. Those are done in the corresponding wrappers by macro THREAD_FREEZE_POINT*.

Later when a child process is forked, RecreateThreads() iterates through the linked list sAllThreads in child process and invokes pthread_create() on each TINFO_FLAG_NUWA_SUPPORT flagged one to recreate them. The startup function here is thread_recreate_startup(), which restores thread name, remember recreated pthread ID and native thread ID, setjmp(), and then longjmp() to the place where the thread setjmp() earlier in Nuwa process to continue the thread. This is also why a malloced stack is used, it is necessary for keeping the stack and resuming the thread in child process. After the thread routine terminates, it returns to thread_create_startup() and then longjmp() back to where it setjmp() in thread_recreate_startup(). Here's an image for better understanding:

You may notice there are some other wrapped pthread functions as well, for example: pthread_join(). It is wrapped because the ID of pthread created in Nuwa process may be stored somewhere and inherited in child process. If child process calls pthread_join() with the ID, it needs to map to the recreated one, otherwise there'll be an error.

I hope this shed some light. :)

[1] http://en.wikipedia.org/wiki/N%C3%BCwa
[2] http://ftp.gnu.org/pub/old-gnu/Manuals/ld-2.9.1/html_node/ld_3.html
[3] http://hg.mozilla.org/mozilla-central/file/7297cfffd91c/mozglue/build/Nuwa.cpp

The first patch on m-c

https://hg.mozilla.org/mozilla-central/rev/78e0fd0ed33f

Draft - Debug Firefox memory leak

Last week Kyle Huey gave a talk about debugging memory leak, this's the note I took, still in draft, will complete it later. Check also Kyle's blog for the same topic.

# Get a debug build Firefox
  $ hg clone https://hg.mozilla.org/mozilla-central
  $ cd mozilla-central
  $ ./mach bootstrap
  $ echo 'ac_add_options --enable-debug' > mozconfig
  $ ./mach build

# Used bug 847138 as an example. Open the test case i.svg, close it. Open
  another tab for about:memory, minimize memory usage, and measure can see the
  leak:

  ├──0.37 MB (00.32%) -- top(none)/detached/window(file:///home/ting/w/fx/i.svg)
  │  ├──0.35 MB (00.31%) ++ js-compartment(file:///home/ting/w/fx/i.svg)
  │  ├──0.01 MB (00.01%) ++ dom
  │  ├──0.01 MB (00.01%) ── style-sheets
  │  └──0.00 MB (00.00%) ── property-tables

# Save GC & CC logs from about:memory
  < cc-edges.xxxxx.xxxxxxxxxx.log >
    > 0x7f3852047100 globalE
    0x7f3852047100 [gc] JS Object (SVGZoomEvent)
    > 0x7f385202a220 type_proto
    > 0x7f385204dbc0 getter
    > 0x7f386ae43240 parent
    > 0x7f385d9f6c10 UnwrapDOMObject(obj)
    0x7f385d9f6c10 [rc=1] Event
    > 0x7f385d74d000 mPresContext
    > 0x7f3851df1800 mOwner
    > 0x7f3859d09800 mView

  < gc-edges.xxxxx.xxxxxxxxxx.log >
    > 0x7f3852047100 G globalE
    0x7f38556cbd20 B string globalE

# $ git clone https://github.com/amccreight/heapgraph.git
  $ cd heapgraph
  $ python find_roots.py /tmp/cc-edges.11209.1396360888.log nsDocument | less

  0x7f385d7fbce0 [nsISVGPoint]
      --[Preserved wrapper]--> 0x7f3852047140 [JS Object (SVGPoint)]
      --[type_proto]--> 0x7f385202a250 [JS Object (SVGPointPrototype)]
      --[getter]--> 0x7f385204dcc0 [JS Object (Function - y)]
      --[parent]--> 0x7f386ae43240 [JS Object (Window)]
      --[document]--> 0x7f3852047040 [JS Object (SVGDocument)]
      --[UnwrapDOMObject(obj)]--> 0x7f385d74c000 [nsDocument normal ([none]) file:///home/ting/Desktop/i.svg]

      Root 0x7f385d7fbce0 is a ref counted object with 1 unknown edge(s).
      known edges:
         0x7f3852047140 [JS Object (SVGPoint)] --[UnwrapDOMObject(obj)]--> 0x7f385d7fbce0
 
  < cc-edges.xxxxx.xxxxxxxxxx.log >
    0x7f385d7fbce0 [known=1]
    0x7f385d7fbce0 [rc=2] nsISVGPoint
    > 0x7f3852047140 Preserved wrapper
    0x7f3852047140 [gc] JS Object (SVGPoint)
    > 0x7f385202a250 type_proto
    > 0x7f386ae43240 parent
    > 0x7f385d7fbce0 UnwrapDOMObject(obj)

# $ python ./tools/heapgraph/find_roots.py ./gc-edges.3131.1396402800.log SVGZoomEvent | less

  via Preserved wrapper :
  0x7f3852047140 [SVGPoint ]
      --[shape]--> 0x7f385204ee98 [shape]
      --[base]--> 0x7f385204f628 [base_shape]
      --[parent]--> 0x7f386ae43240 [Window 7f385b5f4260]
      --[globalE]--> 0x7f3852047100 [SVGZoomEvent ]

  Found and displayed 1 paths.
  via Preserved wrapper :
  0x7f3852047140 [SVGPoint ]
      --[shape]--> 0x7f385204ee98 [shape]
      --[base]--> 0x7f385204f628 [base_shape]
      --[parent]--> 0x7f386ae43240 [Window 7f385b5f4260]
      --[globalE]--> 0x7f3852047100 [SVGZoomEvent ]
      --[type]--> 0x7f385203e8c8 [type_object]
      --[type_proto]--> 0x7f385202a220 [SVGZoomEventPrototype ]

  Found and displayed 1 paths.

Ref - The C++/Programing books

Guess I will need to read some: The C++/Programming books I recommend.

How to add a link attachment to bugzilla

Learned this from my colleague Yoshi: upload a html file with content:
Now click on the attachment, the html will be loaded and redirect to your page. The href is added in case if redirect does not happen.

I was trying to add a url of one google doc as an attachment to bugzilla, but found it's not as easy as I thought. Firstly I pasted the url as an attachment, which bugzilla created a text file with mime type "text/plain" initially, but click on the attachment doesn't redirect to the doc. I then tried different mime types for the text file: "text/url", "text/x-url", and "application/x-url" but none of them work.

Use awk in shell

This's what Thinker typed on my computer today:

  $ adb shell kill -SIGCONT $(adb shell b2g-ps|grep Homescreen|awk '{print $3;}')

The first post

https://bugzilla.mozilla.org/show_bug.cgi?id=945174#c19

Dream BIG, and move humanity forward!

We have lots of "computing" devices, why waste?