Tumblelog by Soup.io
Newer posts are loading.
You are at the newest post.
Click here to check if anything new just came in.

May 15 2020

14:57

Lecture on Augmented Reality

Due to the current circumstances, I had to record the lectures on augmented reality, which I am typically holding live. This was much more work than anticipated..
On the other hand, this means that I can make them available via Youtube now.

So, if you ever wanted to learn about the basic algorithms behind Augmented Reality, now is your chance.

The lecture is structured in two parts

  • Camera Calibration, and
  • Object Pose Estimation.

You can click on the TOC links below to directly jump to a specific topic.

Camera Calibration

Object Pose Estimation

0 Add to favourites0 Bury

April 24 2020

15:07

Error handling and exceptions

Yes, this is yet another post in the internet talking about using exceptions versus error returns. The topic has been flaming up at my workplace for quite some time now, and I felt that writing a blog post about it during the week-end would help me focus my thoughts and give me time to explain my point with the due care. In case you didn't know, I'm against using exceptions for error handling (maybe having spent many years working with Qt has had an effect on this); that does not mean that I never write code using exceptions: I certainly do my good share of try ... catch when dealing with third-party code (including the STL), but you won't find a throw in my programs.

I'm not going to write here all the reasons why I refrain myself from implementing error handling using exceptions; I'd rather like to focus on the one I consider to be the major one, and which I rarely see being given the due weight in the debate.

Code safety

I was about to title this “Code readability”, but this is more about code verifiability, that is making sure that the code is correct and, ultimately, safe. As we all know, code is written once but read many times, and even if it's code you've written yourself, chances are that in a few weeks time you'll have forgotten several details about it; error cases and error handling are one typical thing that doesn't stick in our memory for long.

When I look at a small piece of code, such as the one that can fit into my screen, or which I can read from a merge request diff, I want to be able to ascertain that the code I'm looking at is correct. Let's look at some examples.

A throw-free project

assert(track != nullptr); Car car; car.setMaximumSpeed(90); car.setName("Herbie"); if (!car.executeLap(track)) {     log("Car failed to complete track");     return false; } Path *path = car.getPath(); if (!path) {     log("GPX path could not be retrieved");     return false; } double temperature = car.engineTemperature(); double boundingRectArea = path->boundingRectArea();

I just made this up, so please bear with me if it doesn't make any sense. What I want to show is that code like the above has very few fault risks, if found in a project which bans throwing errors as exceptions: if we exclude out-of-memory errors, that are generally not handled to let the application crash (though you can always catch them if you like), the reader can easily verify that this code is safe. Coding style policies and naming conventions can guarantee that setMaximumSpeed() and setName() won't have a return value that needs to be checked, and all other method calls either return an error that our code is properly handling, or return some value. Of course, by just looking at this piece of code we cannot know if the engineTemperature() method has some other overloaded sibling which accepts passing a reference to a boolean and which could be used to detect an error; so, it may be that our code could be improved in that respect, if we had a look at the header files for the Car class — but this does deny the fact that a simple glance at this snippet tells us exactly what errors are handled and what could be going wrong.

Let's look at this code instead:

assert(track != nullptr); Car car; car.setMaximumSpeed(90); car.setName("Herbie"); car.executeLap(track); Path *path = car.getPath(); double temperature = car.engineTemperature(); double boundingRectArea = path->boundingRectArea();

If we continue on the assumption that we are working on a project which bans throwing exceptions, we can immediately say that this code is not safe: we don't know if the car successfully executed a lap on the track, and our process will crash if boundingRectArea() is invoked on a null object.

Enter the exception

In a project where exceptions are actively used, the code from the second snippet is not obviously wrong anymore: maybe executeLap() cannot throw any exceptions, or, if does, the caller of this snippet is catching the exception? In order to figure out whether this code is correct, I need to see the declaration of the executeLap() method, and hope that there's a nice noexcept in there; if there isn't, I have to look at its implementation, and recursively descend through all the methods it calls — at which point the safest attitude is just to assume that it can throw. But that's only half of the story, because once I accept the fact that executeLap() can throw, I need to check whether the exception is properly handled: I have to check the implementation of all the callers of my method, and if I don't find a catch there, I'll have to recursively walk up the tree of their callers.

And indeed even the first snippet, which looked so harmless when exception throwing was banned, suddenly becomes not obviously correct anymore: what if executeLap() or getPath() also throw an exception? You might say that it would be quite a silly thing to do, and I'd certainly agree; but it may be that indeed they don't throw any exceptions in their implementation, but some of the methods they call does.

A compromise: catch early, catch often

The obvious solution to the above issue is having a policy of handling exceptions right away, and explicitly rethrowing them (or even better, rethrow a different, more appropriate exception) up the stack:

assert(track != nullptr); Car car; car.setMaximumSpeed(90); car.setName("Herbie"); try {     car.executeLap(track);     Path *path = car.getPath();     double temperature = car.engineTemperature();     double boundingRectArea = path->boundingRectArea(); } catch (std::runtime_error &e) {     log("Car failed to complete track");     throw; }

What I can tell from the above snippet is that the code is handling errors, and this is somehow a relief. I'm sure some of you would suggest using a more specific catch clause, but for the sake of this example let's assume that this one is fine.

(Quick note: the above example does not catch std::exception, because that would also catch the std::bad_alloc exception which is typically thrown in out-of-memory situations; my advice is not handle it at all, unless you know what you are doing)

In real life, though, you might find that try-ing on a rather large block of operations is not enough: suppose that the Car methods all emit the same exception type, and that you need to handle them differently depending on when they occur. Then you'd need to split up the try into smaller blocks, and at that point your code won't look any cleaner than the equivalent code which uses ifs on return values. Of course if you own the Car class you could modify it to throw different exceptions, in order to keep more operations inside the try block and have specific catches at the end.

The big catch (pun intended)

Even once you've refactored your methods to get the best out of exceptions (where "best" is highly subjective, but let's assume that it just means that you are happy with your exception-throwing code), there's something that still bothers me, and that's exactly the same thing that proponents of exceptions use as a “pro” in their argumentations: the business logic of your code gets separated from the error handling. You get a nice block of pure logic, not cluttered with error checking, and a catch section (which I call “the big catch”) where error cases are handled.

I really don't see how that makes the code any more readable or safe: sure, the logic is not intertwined with error handling and might help focus on the expected flow of the operations (though, really, I do not think that normal brains have a problem skipping over if blocks), but that's hardly what I'm interested in when I want to check that the code is correct. Most of program errors and bugs lie in handling the edge cases and the abnormal situations, the seldomly taken code paths, and that's where I need to focus my attention.

try {     operationA();     if (value > B.maxValue()) {         operationB();     } else {         operationC();     }     operationD(); } catch (ExceptionI &e) {     ... } catch (ExceptionII &e) {     ... } catch (ExceptionIII &e) {     ... } catch (std::runtime_error &e) {     ... }

When I see code like this one, I need to mentally build a mapping of “operationX() → possible exceptions” (which, unless exception naming is making this obvious, requires me to look at the implementation of the operationX() functions), and then mentally reconstruct the possible code paths in case operationX() fails, for each line of the try block.

Not seeing the errors right there, right away makes the correctness verification harder, which in turns means that the code becomes less safe. It will make you focus on the best case scenario, while ignoring all those annoying edge cases — too bad that 90% of the bugs are there.

Reading through the ISO C++ propaganda FAQ

I've been given a link to the C++ FAQ about exceptions, and unfortunately I read it. While there isn't much to argue on the technical side of it, it also carries some misleading statements, which might be true in absolute terms but don't let you see the big picture by not mentioning all that you need to know (which is the fundamental technique behind propaganda). An example is when they mention that eliminating ifs makes for more robust code, without mentioning that the same applies to all code branches, including exceptions.

Another argument that bothered me when I read it is the one about error propagation; this is the example they make:

void f1() {     try {         // ...         f2();         // ...     } catch (some_exception& e) {         // ...code that handles the error...     } } void f2() { ...; f3(); ...; } void f3() { ...; f4(); ...; } void f4() { ...; f5(); ...; } void f5() { ...; f6(); ...; } void f6() { ...; f7(); ...; } void f7() { ...; f8(); ...; } void f8() { ...; f9(); ...; } void f9() { ...; f10(); ...; } void f10() {     // ...     if ( /*...some error condition...*/ )         throw some_exception();     // ... }

The claim is that this code is more readable than the one with explicit error handling, because all the f2(), f3, …, f9() functions don't have to handle the error occurring in f10(). It is indeed a convincing argument, when presented in these terms, but is this really how our code looks like? In real life, you'll hardly have a chain of 1-liner functions, all defined next to each other in the same file. The moment that you realize that each one of these fn() functions might be twenty or thirty lines long, and that they might be scattered over different files, and be called not just by fn-1() but by any other function in the codebase, the picture does not look so rosy anymore: we get back to my main point of pain, that is that looking at the code of, say, f5(), I will not be able to tell if the errors thrown by it, or by any of the methods invoked by it, are properly handled.

Exceptions in APIs

A side note about projects using exceptions. I'm not really bothered when a library I need to use is throwing exceptions: having to write

try {     Foo::fetch("http://example.com/resource.txt"); } catch (Foo::Exception &) {     return false; }

is not less readable or less safe than the code I'd write if Foo::fetch() returned an error code. I still do have a little complaint, because the library author has given himself the right to decide that a failure in his library should be considered a critical fault, whereas it may be that in my program it is an expected failure and using exceptions imposes a penalty which could have been avoided. But I digress.

As long as the library documents which exceptions are thrown, it is used by many people (which hopefully means that it has few bugs) and it is a library that I don't need to contribute to, wrapping some of its methods in try blocks is something I can live with.

One situation where I actually wish that libraries threw an exception is in out-of-memory situations; in that case, of course, I'd expect them to throw nothing else than std::bad_alloc, which is the exception emitted by the standard library in such situations. That allows the caller to decide whether to ignore the exception and have the process terminated (which is what I usually do, at least in desktop applications) or try their luck and handle the failure — the latter is not easy, but it can certainly be done.

This is one case where error returns can be problematic, because it's likely that your code would look something like

if (!Foo::open(fileName)) {   // suppose that this returns Error::OutOfMemory     log("Failed to open " << fileName);     return false; }

and in this case there's actually a risk that your code is going to trigger an out-of-memory error in logging the message; this shouldn't be a concern in most cases, but I can imagine some situations where one might want to know which was the exact operation that first incurred in the out-of-memory failure.

So, I'm actually fine with new throwing. As for my code, my throw statement is actually spelt as return.

0 Add to favourites0 Bury
Sponsored post
Reposted bySchrammelhammelMrCoffeinmybetterworldkonikonikonikonikoniambassadorofdumbgroeschtlNaitliszpikkumyygittimmoejeschge

April 12 2020

09:20

New website for Mappero Geotagger, and cross-compiling stuff

Mappero Geotagger has now moved from its previous page from this site to a new, separate website built with the awesome Nikola static website generator.

The main reason for this change is that I didn't have an online space where to host the application binaries, and I wanted to experiment with a different selling method. Now, downloads are (poorly) hidden behind a payment page, whereas in multiple places of the website I also mention that I can provide the application for free to whomever asks for it. While it might seem weird at first, I do honestly believe that this will not stop people from buying it: first of all, many people just think it's fair to pay for a software applications, and secondly, for some people writing an e-mail and establishing a personal contact with a stranger is actually harder than paying a small amount of money. And in all sincerity, the majority of the income I've had so far for Mappero Geotagger came from donations, rather than purchases; so, not much to lose here.

QBS and MXE

Anyway, since this is primarily a technical blog, I want to share my experiences with cross-building from Linux to Windows. As you might remember, some time ago I switched the build system of Mappero from qmake to QBS, and I haven't regretted it at all. I've managed to build the application in Linux (of course), macOS, as a Debian package on the Ubuntu PPA builders, on Windows with AppVeyor and, last but not least, on Linux for Windows using the mingw setup provided by the MXE project.

QBS worked surprisingly well also in this case, though I had to fight with a small bug on the toolchain detection, which is hopefully going to be fixed soon. For the few of you who are interested in achieving something similar, here's the steps I ran to configure QBS for mingw:

    MXE_BASE=<path-to-mxe>     MXE_TARGET=x86_64-w64-mingw32.shared # 32 bit or static targets are also available     MXE_PROFILE="mxe"     QT_PROFILE="${MXE_PROFILE}-qt"     qbs setup-toolchains "${MXE_BASE}/usr/bin/${MXE_TARGET}-g++" $MXE_PROFILE     qbs config profiles.$MXE_PROFILE.cpp.toolchainPrefix "${MXE_TARGET}-" # temporary workaround     qbs setup-qt "$MXE_BASE/usr/$MXE_TARGET/qt5/bin/qmake" ${QT_PROFILE}     qbs config profiles.${QT_PROFILE}.baseProfile $MXE_PROFILE

Sorry for using that many environment variables ☺. After qbs is configured, it's just a matter of running

    qbs profile:$QT_PROFILE

to build the application. You will get a nice window binary and, once you collect all the needed library dependencies, you'll be able to run it on Windows. Or WINE ☺.

As part of this effort, I also had to build libraw, so I didn't miss the occasion to contribute its recipe to MXE. I'm also trying to get a change accepted, that would make MXE support the dynamic OpenGL selection available since Qt 5.4.

0 Add to favourites0 Bury

March 23 2020

11:40

Are we dead yet?

I am quite frustrated with corona graphs in the news, since most reporters seem to have skipped math classes back then. For instance, just plotting the number of confirmed infections at the respective dates does not tell you anything due to the different time point of outbreak. So lets see whether I can do better:

https://paroj.github.io/arewedeadyet/

With the site above, I tried to improve on a few things:

  • the Charts are live: they update themselves each time you load the site.
  • The curves are normalized by the time-point of outbreak so you can compare the course in different countries.
  • You can select the countries that you want to compare.
  • Different metrics are computed that allow comparing the corona countermeasures and impact across countries with different population size.
0 Add to favourites0 Bury

March 09 2020

07:13

Need a fast way to tag faces in many images? Try Imaginario!

Today I've released Imaginario 0.9. The big feature coming with this new release is a face tagging flow which I believe will be the fastest and simplest you've ever used, despite it being all manual. I even sat down and spent some quality time with Blender to prepare a video to show it off:

While some people might actually think that I spent more time for making the video than for implementing the face tagging feature itself, this couldn't be farther from the truth: the face tagging branch has been being worked on for at least three months (of course, that's my spare time — so it's actually less than one hour per day) and consisted of more than 40 commits (after squashing all the fixups), whereas for the video I spent no more than a couple of hours.

I would appreciate if the curious could go and try it out, and let me know about any issues you should find: there are built packages for Linux (AppImage), macOS and Windows. I do also have an Ubuntu PPA where nightly images are built, but I'm not sure if I can recommend that one, since I've not been using it myself and have no idea whether those packages actually even start. But you are welcome to try :-)

Your feedback will help me do better, so please don't be shy!

0 Add to favourites0 Bury

January 28 2020

14:57

TheQtCompany and a silly decision waiting to be reverted

TheQtCompany has announced a series of changes in their Qt offering which sound a lot like some previously reverted changes, just worse.

I'm going to state my opinion as well, of course. There have been several messages with criticism both in the Qt developers' mailing list as well as in various blogs (including some hosted in Planet KDE), and among this loud, predictable chorus of complaints there are a few messages that, in my opinion, shine for clarity and insightfulness. One is the blog post from Boudewijn Rempt, one of the main developers of the highly successful Krita application. Then, speaking about the issue of Qt LTS releases, Giuseppe d'Angelo posed a few still unanswered questions in the mailing list. The most frightening reply of all is Nikolai Marchenko's message about TheQtCompany's presumed business model, to which I cannot really find any strong counter-arguments.

What is left to me to say? Well, the first thing I'd like to say is that, as a Qt contributor with a history of several upstream contributions many of which developed in my own free time, this sounds like a big middle finger pointed at me. Or like saying that my effort is not worth the access to a LTS or to a password-free download of binary releases. This is not really encouraging contributions from the community.

The main point, though, is that TheQtCompany is effectively making it hard for developers to start using it and to consider it as a true open source project. Especially other open source projects, like UBports (which is based on Qt LTS releases) are going to feel the pain. It's going to alienate Qt advocates, and at the end reducing the user base.

And the funny thing, and the point that in my opinion TheQtCompany has not pondered well enough, is that Qt is still open source, so this move is actually going to benefit their competitors: suppose that an imaginary company NOKLA (but it could be a very real company, one of those who make a business selling Qt consultancy) sets up open download repositories and offers even minimally maintained LTS branches at qt.nokla.com. What will happen is that the name NOKLA will eventually be associated with "the place where you can download Qt", or "where you can find a well-maintained LTS for Qt", effectively boosting the NOKLA brand and implicitly enshrining them as the company to talk to in order to get your Qt-related business done.

I'm quite sure that TheQtCompany will end up reverting this decision, at least for the most part. My guess is that it's the fruit of the mind of some newly hired manager who understands very little of open source, business or marketing and who has been given a little too much decision power. Time will tell.

0 Add to favourites0 Bury

November 06 2019

16:26

Fast wire-frame rendering with OpenCV

Lets say you have mesh data in the typical format, triangulated, vertex buffer and index buffer. E. g. something like

>>> vertices

[[[ 46.27500153  19.2329998   48.5       ]]

 [[  7.12050009  15.28199959  59.59049988]]

 [[ 32.70849991  29.56100082  45.72949982]]

 ..., 

>>> indices

[[1068 1646 1577]
 [1057  908  938]
 [ 420 1175  237]
 ..., 

Typically you would need to feed it into OpenGL to get an image out of it. However, there are occasions when setting up OpenGL would be too much hassle or when you deliberately want to render on the CPU.

In this case we can use the OpenCV to do the rendering in two function calls as:

img = np.full((720, 1280, 3), 64, dtype=np.uint8)

pts2d = cv2.projectPoints(vertices, rot, trans, K, None)[0].astype(int)
cv2.polylines(img, pts2d[indices], True, (255, 255, 255), 1, cv2.LINE_AA)

See the documentation of cv2.projectPoints for the meaning of the parameters.

Note how we only project each vertex once and only apply the mesh topology afterwards. Here, we just use the numpy advanced indexing as pts2d[indices] to perform the expansion.

This is pretty fast as well. The code above only takes about 9ms on my machine.

In case you want filled polygons, this is pretty easy as well

for face in indices:
    cv2.fillConvexPoly(img, pts2d[face], (64, 64, 192))

However, as we need to a python loop in this case and also have quite some overdraw, it is considerable slower at 20ms.

Of course you can also combine both to get an image like in the post title.

From here on you can also go crazy and compute a face normal to do hidden surface removal and flat shading.

0 Add to favourites0 Bury

November 02 2019

15:56

Xiaomi AirDots Pro 2 / Air2 Review

So after having made fun of people for “wearing toothbrushes”, I finally came to buy such headphones for myself.

Having used non-true wireless Bluetooth headphones before I was curious what the usability advantage would feel like.

Here I went for the Xiaomi AirDots Pro 2 aka Air2 which I could grab for 399 Yuan which is about 51€, which seems like the right price-point for this kind of accessoire.

Keep in mind that the built-in battery only survives so many charging cycles and once it dies you can throw them away.

The Airdots (right) compared to the Airpods (middle) and the Oral-B Precision Clean

The initial feeling of using true wireless headphones is surprisingly relieving – there is simply no cord to untangle or to be aware of while wearing.
This is especially true during phone calls, where one needs to keep the microphone aligned.

The downside is that the headphones are too small to accommodate any buttons for volume and playback control.

The Air 2 kind of make up for it by automatically connecting to your phone once you put them on and by automatically pausing the music when you put one out of the ear. This is achieved by a built-in brightness sensor.

Furthermore you have double-tap actions, which default to play/ pause on the right headphone and launching the voice assistant (e.g. Google Assistant) on the left headphone.

The battery life is stated with 4 hours per-charge with 2 extra charges in the case. I could confirm those on a long distance flight.

Compared to the Airpods 2

Looking at the feature-list above or simply at the images, the similarity to the Apple Airpods is apparent.

Out of curiosity I borrowed some from friend for comparison. The most important point is probably sound quality. Here we found the two virtually in-distinguishable. But keep in mind that we only did a quick test and did not use them extensively.

The second point is likely the form. Here, both earphones have the same ear-part and only differ by the shaft. So if one fits your ear, so should the other.

The shaft however is considerably wider on the Airdots. This is less apparent when viewed from the side as the thickness is similar.

For me, the more important difference is being able to control the headphones from my Android smartphone. This is currently not possible with the Airpods, while there is some way for the Airdots;

Companion app & Software integration

To control the earphones, you have to sideload the Xiao Ai Lite App. The main purpose of it is to provide the Xiaomi voice assistant and the Air2 options likely just ended up there as they offer an always-on assistant integration just like the Airpods.

It allows you to choose the douple-tap action per earphone and see the charging status of the individual earphones and the case separately.

The downside is that the app is currently only available in Chinese and consequently the voice assistant only works with Chinese.

Below you find some instant-translated views from the settings pane.

I tried out some voice commands via google translate and everything works as it should. However if you are not fluent in chinese it is far from practical. Most people should disable the assistant in the settings to avoid accidentally triggering it.

If you own a Xiaomi Phone you get those settings via MIUI as well as a slightly faster pairing process. Here, instead of pressing the pairing button, you only have to open the case while holding it near your phone.

A serious advantage of Xiomi/ Huawei phones is the availability of the LHDC Bluetooth Codec which offers a superior bandwidth and latency.
While I am fine with the bandwidth provided by AAC when listening to music, there is still a noticeable and annoying delay when watching videos and playing games.

0 Add to favourites0 Bury

October 13 2019

18:51

Implementing "Open with…" on MacOS with Qt

I just released PhotoTeleport 0.12, which includes the feature mentioned in the title of this blog post. Given that it took me some time to understand how this could work with Qt, I think it might be worth spending a couple of lines about how to implement it.

In the target application

The first step (and the easiest one) is about adding the proper information to your .plist file: this is needed to tell MacOS what file types are supported by your application. The official documentation is here, but given that an example is better than a thousand words, here's what I had to add to PhotoTeleport.plist in order to have it registered as a handler for TIFF files:

  <key>CFBundleDocumentTypes</key>
  <array>
    <dict>
      <key>CFBundleTypeExtensions</key>
      <array>
        <string>tiff</string>
        <string>TIFF</string>
        <string>tif</string>
        <string>TIF</string>
      </array>
      <key>CFBundleTypeMIMETypes</key>
      <array>
        <string>image/tiff</string>
      </array>
      <key>CFBundleTypeName</key>
      <string>NSTIFFPboardType</string>
      <key>CFBundleTypeOSTypes</key>
      <array>
        <string>TIFF</string>
        <string>****</string>
      </array>
      <key>CFBundleTypeRole</key>
      <string>Viewer</string>
      <key>LSHandlerRank</key>
      <string>Default</string>
      <key>LSItemContentTypes</key>
      <array>
        <string>public.tiff</string>
      </array>
      <key>NSDocumentClass</key>
      <string>PVDocument</string>
    </dict>
    …more dict entries for other supported file formats…
  </array>

This is enough to have your application appear in Finder's "Open with…" menu and be started when the user selects it from the context menu, but it's only half of the story: to my big surprise, the selected files are not passed to your application as command line parameters, but via some MacOS-specific event which needs to be handled.

By grepping into the Qt source code, I've found out that Qt already handles the event, which is then transformed into a QFileOpenEvent. The documentation here is quite helpful, so I won't waste your time to repeat it here; what has hard for me was to actually find that this functionality exists and is supported by Qt.

In the source application

The above is only half of the story: what if you are writing an application which wants to send some files to some other application? Because of the sandboxing, you cannot just start the desired application in a QProcess and pass the files as parameters: again, we need to use the Apple Launch Services so that the target application would receive the files through the mechanism described above.

Unfortunately, as far as I could find this is not something that Qt supports; sure, with QDesktopServices::openUrlExternally() you can start the default handler for the given url, but what if you need to open more than one file at once? And what if you want to open the files in a specific application, and not just in the default one? Well, you need to get your hands dirty and use some MacOS APIs:

#import <CoreFoundation/CoreFoundation.h>
#import <ApplicationServices/ApplicationServices.h>

void MacOS::runApp(const QString &app, const QList<QUrl> &files)
{
    CFURLRef appUrl = QUrl::fromLocalFile(app).toCFURL();

    CFMutableArrayRef cfaFiles =
        CFArrayCreateMutable(kCFAllocatorDefault,
                             files.count(),
                             &kCFTypeArrayCallBacks);
    for (const QUrl &url: files) {
        CFURLRef u = url.toCFURL();
        CFArrayAppendValue(cfaFiles, u);
        CFRelease(u);
    }

    LSLaunchURLSpec inspec;
    inspec.appURL = appUrl;
    inspec.itemURLs = cfaFiles;
    inspec.asyncRefCon = NULL;
    inspec.launchFlags = kLSLaunchDefaults + kLSLaunchAndDisplayErrors;
    inspec.passThruParams = NULL;

    OSStatus ret;
    ret = LSOpenFromURLSpec(&inspec, NULL);
    CFRelease(appUrl);
}

In Imaginario I've saved this into a macos.mm file, added it to the source files, and also added the native MacOS libraries to the build (qmake):

LIBS += -framework CoreServices

You can see the commit implementing all this, it really doesn't get more complex than this. The first parameter to the MacOS::runApp() function is the name of the application; I've verified that the form /Applications/YourAppName.app works, but it may be that more human-friendly variants work as well.

0 Add to favourites0 Bury

October 04 2019

08:36

Bussator: implementing webmentions as comments

Recently I've grown an interest to the indieweb: as big corporations are trying to dictate the way we live our digital life, I'm feeling the need to take a break from at least some of them and getting somehow more control over the technologies I use.

Some projects have been born which are very helpful with that (one above all: NextCloud), but there are also many older technologies which enable us to live the internet as a free distributed network with no owners: I'm referring here to protocols such as HTTP, IMAP, RSS, which I perceive to be under threat of being pushed aside in favor of newer, more convenient, but also more oppressive solutions.

Anyway. The indieweb community is promoting the empowerment of users, by teaching them how to regain control of their online presence: this pivots arund having one's own domain and use self-hosted or federated solutions as much as possible.

One of the lesser known technologies (yet widely used in the indieweb community) is webmentions: in simple terms, it's a way to reply to other people's blog posts by writing a reply in your own blog, and have it shown also on the original article you are replying to. The protocol behind this feature is an recommendation approved by the W3C, and it's actually one of the simplest protocol to implement. So, why not give it a try?

I already added support for comments in my blog (statically generated with Nikola) by deploying Isso, a self-hosted commenting system which can even run as a FastCGI application (hence, it can be deployed in a shared hosting with no support for long-running processes) — so I was looking for a solution to somehow convert webmentions into comments, in order hot to have to deal with two different commenting systems.

As expected, there was no ready solution for this; so I sat down and hacked up Bussator, a WSGI application which implements a webmention receiver and publishes the reply posts as Isso comments. The project is extensible, and Isso is only one of the possible commenting systems; sure, at the moment it's indeed the only one available, but there's no reason why a plugin for Static Man, Commento, Remark or others couldn't be written. I'll happily accept merge requests, don't be shy — or I can write it myself, if you convince me to (a nice Lego box would make me do anything 0 Add to favourites0 Bury

September 17 2019

16:38

On Richard Stallman and people who cannot read

We live in strange times. People are so filled with hatred and prejudices that their brain becomes unable to parse the simplest sentences. I take this issue to heart, because it could happen to anyone — it has happened to me before (luckily, only in private online conversations), where an acquaintance of mine accused me of saying things I never said. And it happens to famous people all the time. Guys, just because you hate person X, you should not skip over parts of their speech or suppress context in order to make it look like they said something terrible or stupid, when they didn't.

Now it happend to Richard Stallman, with a whole wave of hateful people accusing him of saying something that he didn't say. Let's start with the VICE article, titled "Famed Computer Scientist Richard Stallman Described Epstein Victims As 'Entirely Willing'", which insists in quoting only two words out of Stallman's sentence:

Early in the thread, Stallman insists that the “most plausible scenario” is that Epstein’s underage victims were “entirely willing” while being trafficked.

Except that he didn't say that. Why not quote the whole sentence? It's not such a long sentence, really! Just follow the link to the source, which provides a complete excerpt of Stallman's words:

We can imagine many scenarios, but the most plausible scenario is that she presented herself to him as entirely willing. Assuming she was being coerced by Epstein, he would have had every reason to tell her to conceal that from most of his associates.

Now, English is not my native language, but I read it well enough to understand that “to present oneself as” and “to be” are different expressions having very different meanings (and, in most context, actually opposite ones!). You don't need to be Shakespeare to understand that. You only need to either stop hating or, if you really cannot help it, at least stop projecting your prejudices onto the people you hate. Hate makes you blind.

It's sad to see otherwise intelligent people take stupid decisions because of such misunderstandings.

I for one, stand in solidarity with Richard Stallman and with the English language.

(please note that this is not an endorsement of everything Stallman might have said in the past; I don't follow him that closely, and it may be that he also happened to say terrible things in this very thread. I'm only commenting this very specific issue, and I know that in this very specific issue he's being wrongly accused)

0 Add to favourites0 Bury

August 20 2019

14:39

Migrating to a new Mastodon instance

The wonders of improvised Mastodon instances: one node disappears after an outage caused by a summer heatwave, leaving its users no way to migrate their data or to notify their followers.

After about one month of waiting for the node to come up or give some signals of life, I've decided to create a new account on another instance. If you use Mastodon and you were following me, please forgive me for the annoyance and mardy">follow me again here.

0 Add to favourites0 Bury

August 09 2019

16:13

calibDB: easy camera calibration as a web-service

This image has an empty alt attribute; its file name is overlay1.jpg

Camera calibration just got even easier now. The pose calibration algorithm mentioned here is available as web-service now.

This means that calibration is no longer restricted to a Linux PC – you can also calibrate cameras attached to Windows/ OSX and even mobile phones.
Furthermore you will not have to calibrate at all if your device is already known to the service.
The underlying algorithm ensures that the obtained calibrations are reliable and thus can be shared between devices of the same series.

Aggregating calibrations while providing on-the-fly calibrations for unknown devices form the calibDB web-service.

In the future we will make our REST API public so you can transparently retrieve calibrations for use with your computer vision algorithms.
This will make them accessible to a variety of devices, without you having to worry about the calibration data.

0 Add to favourites0 Bury
15:45

Beyond the Raspberry Pi for Nextcloud hosting

When using Nextcloud it makes some sense to host it yourself at home to get the maximum benefit of having your own cloud.

If you would use a virtual private server or shared hosting, your data would still be exposed to a third party and the storage would be limited as you would have to rent it.

When setting up a server at home one is tempted to use a Raspberry Pi or similar ARM based device. Those are quite cheap and only consume little power. Especially the latter property is important as the machine will run 24/7.

I was as well tempted and started my self-hosting experience with an ARM based boards, so here are my experiences.

Do not use a Raspberry Pi for hosting

Actually this is true for any ARM based board. As for the Pi itself, only the most recent Pi 4B has a decent enough CPU and enough RAM to handle multiple PHP request (WebCAL, Contacts, WebDAV) from different clients without slowdown.
Also only with the Pi 4B you can properly attach storage over USB3.0 – previously your transfer rates would be limited by the USB2.0 bus.

One might argue that other ARM based computers are better suited. Indeed you could get the decently equipped Odroid U3, long before the Pi 4B was available.
However, non-pi boards have their own set of problems. Typically, they are based on an Smartphone design (e.g. the Odroid U3 essentialy is a Galaxy Note 2).

This makes them plagued by the Android update issues, as these boards require a custom kernel, that includes some of the board specific patches which means you cannot just grab an Ubuntu ARM build.
Instead you have to wait for a special image from the vendor – and just as with Android, at some point, there will be no more updates.

Furthermore ARM boards are actually not that cheap. While the Pi board itself is indeed not expensive at ~60€, you have to add power-supply housing and storage.

Intel NUC devices are a great choice

While everyone was looking at cheap and efficient ARM based boards, Intel has released some great NUC competitors.
Those went largely unnoticed as typically only the high-end NUCs get news coverage. It is more impressive to report how much power one can cram into a small form-factor.

However one can obviously also put only little power in there. More precisely, Intels tablet celeron chips that range around 4-6W TDP and thus compete with ARM boards power-wise. (Still they are an order of magnitude faster then a Raspberry Pi)

Device Power (Idle) Power (load) Odroid U3 3.7 W 9 W GB-BPCE-3350C 4.5 W 9.6 W

Here, you get the advantages of the mature x86 platform, namely interchangeable RAM, interchangeable WiFi modules, SATA & m2 SSD ports and notably upstream Linux compatibilty (and Windows for that matter).

As you might have guessed by the hardware choice above, I made the switch already some time ago. On the one hand you only get reports for the by now outdated N3350 CPU – but on on the other hand it makes this a long term evaluation.

Regarding the specific NUC model, I went with the Gigabyte GB-BPCE-3350C, which are less expensive (currently priced around 90€) than the Intel models.

Consequently the C probably stands for “cheap” as it lacks a second SO-DIMM slot and a SD-card reader. However it is fan-less and thus perfectly fine for hosting.

So after 2 Years of usage and a successful upgrade between two Ubuntu LTS releases, I can report that switching to the x86 platform was worth it.

If anything I would probably choose a NUC model that also supports M.2/ M-Key in addition to SATA to build a software RAID-1.

0 Add to favourites0 Bury

July 17 2019

14:55

Looking out for ideas: federated / aggregated content

I was looking at TripAdvisor, some days ago. It's a very useful site, filled with user-generated advice and reviews, which has become almost a must for travellers. But I never like it when a private entity gets so much power over our lives (even if it's — currently — exercised in total fairness).

I would like to have a federated TripAdvisor-like network. I duckduckwent for a while, but I didn't find anything similar (if you know of some project of that kind, please let me know in the comments).

But thinking more about the issue, I realised that I probably wouldn't even bother to type my reviews into that site; I have a blog, so ideally I would like to have the option to write my review here, and then have the site import it. Technically, it could work with a webhook, or even a periodic check (real-time updates would not be a requirement here) over a URL I've linked to in my profile on that site. Then, in order for the posts to be imported, they would have to be entered in a standard format: maybe some keywords (or invisible HTML elements) could be used as markers for the content that need to be extracted from an otherwise ordinary blog post, or the relevant content could be replicated in a different format in the HTML headers (this, though, would require some additional work).

And while we are at it, why not extend this to other social networks? I use mardy">Mastodon, for example, and occasionally I send out a toot with a link to my latest blog post in there. But it would be much nicer if I could somehow set a special mark into my posts while composing them, to have them automatically tooted out on my account (this could probably be implemented as a standalone service, authorized to act on my Mastodon account — similarly to how the Mastodon-Twitter crossposter app works).

But I'm rather confident that I'm not the only one having this kind of needs, and that's why I'm writing this blog post: maybe someone out there has already found a solution, or has some more concrete ideas? If so, I'm all eyes!

0 Add to favourites0 Bury

July 04 2019

15:47

Notes on porting the Samsung J3 to Halium + Ubports

pre { white-space: pre !important; overflow-y: scroll !important; max-height: 50vh !important; }

A due premise: this is not a guide! While at the end I managed to build an image for my phone, it doesn't even boot. So, this post should be taken just as a collection of notes for myself, written to be able to reproduce the build and keep track of those steps I might want to revisit later.

The reason why I'm publishing these notes in my blog is that I believe they can still be useful for other people who should stumble into the same errors, and also to give you an idea of how much fun (or lack thereof) porting is.

It's a very long post, but unless you are starting to port Halium or Ubports to a new device, you can safely stop reading now.

I hope to write some better porting news in some future post. :-)

Before we start: getting the info

The first step is finding out a few informations about our device, and whether a LineageOS port for our device exists (hadn't there been a LineageOS port, I would have been out of luck: I certainly wouldn't have the time to do one myself). From the Settings application we can find the device name, the model number, the Android version and the kernel version:

  • Device name: "Samsung Galaxy J3 (2016)"
  • Model: SM-J320F
  • Android version: 5.1.1
  • Kernel version: 3.10.65-[…]

With this information we can look for a LineageOS port, either via a search engine or directly in the XDA forums; in my case the thread is this one. The post announcing the port should contain links to the source code, besides some other information. We do especially care about:

  • the device codename: j3xlte
  • the CM/LineageOS version: 14.1
  • links to Device Tree, Kernel, Local Manifest (keep the local manifest handy, as you are going to need it soon)

To be on the safe side, get a copy of everything. In case these are GitHub repositories, forking them is enough.

Getting root access on the device

Another precondition for a successful port is the ability to be able to flash a custom kernel and rootfs image. The existance of a LineageOS port is not a guarantee that we can flash it: some devices are sold with a locked bootloader in certain markets or when sold by certain mobile operators. So, before spending time on the port, we should make sure that we can hack on our device freely.

In order to get adb working, I had to enable "Developer mode", which is done by hitting the "Build numer" item under the "Software info" page for 10 times in a row. Then from the newly appeared "Developer options" menu I enabled "USB debugging".

Being a total novice to Android phones, I had to go through several attempts and internet searches, before being able to get root access on the device (which I needed in order to read the /etc/fstab file, for example): by default, adb only let me as normal user, and su didn't work.

In order to flash a rootkit on this phone, we need to use Heimdall, because on Samsung devices this is what is used instead of fastboot. I just followed the instructions in the Linux/README file and could easily build and install it.

We are going to flash TWRP and SuperSU; a link to the former can be found in the XDA post about the LineageOS port for our device (under the "Download" button) and the latter from its own homepage (in my case I had to download the "Recovery flashable" zip file, and for some weird reason I got the 2.76 version, but I bet that any newer version works as well). Once I got these files, I followed these steps:

  • Extract TWRP in my host machine: cd /tmp && tar xvf ~/Downloads/sm-j320fgm_twrp_3.1.1_f2fs.tar.md5 (that leaves a recovery.img file in my /tmp)
  • Upload SuperSU to the phone:
    adb push ~/Downloads/UPDATE-SuperSU-v2.76-20160630161323.zip /sdcard/supersu.zip
  • Power off the phone
  • Power it on by keeping the VOLUME DOWN and HOME keys pressed while pressing the power button. The phone will be brought up into Odin mode.
  • Run
    sudo heimdall flash  --RECOVERY /tmp/recovery.img
  • Before the phone reboots, start keeping the VOLUME UP (note: it's not the same as before!) and HOME keys, to make it reboot into recovery and not into the ordinary image. If the phone boots into the ordinary image it will overwrite the TWRP we just flashed, so, if that happens, you need to start from scratch.

  • Find supersu (the zip file) and install it.

If you reboot the phone and use adb shell, you'll be able to run su and get root rights. At this point, I charged myself with hopes and got to the real work.

Let the Odyssey start

I just followed the "First steps" instructions step by step, with no major surprises.

Once done with that, I moved to the "Get sources" page. Here it was not just a matter of following the instructions: assembling the manifest file took me several iterations, so I had to come back to this step a few times. Anyway, I followed the instructions for the halium-7.1 version, given that my device has a LineageOS 14.1 port. Then:

  • I created a git branch in the halium/devices/ tree, to keep track of my changes
  • I added the manifest halium/devices/manifests/samsung_j3xlte.xml (you can see the final version in my repository, but initially I was missing some important lines)
  • Run JOBS=2 ./halium/devices/setup j3xlte
  • The makefile needed to be changed: the device/samsung/j3xlte/setup-makefiles.sh script (from the device tree) was missing a mkdir -p ../../../$OUTDIR near the beginning.

At this point I continued to the "Build sources" page. Here starts a long trial and error phase, and the longer part of this post. For the sake of documentation, I will show you all the commands I ran, including failed attempts (there are many of them), just to give you an idea of what problems you might run into with your own port, and hopefully help in finding a solution.

I first ran the breakfast command for my device:

$ breakfast j3xlte
including vendor/cm/vendorsetup.sh
build/core/product_config.mk:254: *** _nic.PRODUCTS.[[device/samsung/j3xlte/lineage.mk]]: "device/samsung/sharkls-common/sharkls.mk" does not exist.  Stop.
build/core/product_config.mk:254: *** _nic.PRODUCTS.[[device/samsung/j3xlte/lineage.mk]]: "device/samsung/sharkls-common/sharkls.mk" does not exist.  Stop.
build/core/product_config.mk:254: *** _nic.PRODUCTS.[[device/samsung/j3xlte/lineage.mk]]: "device/samsung/sharkls-common/sharkls.mk" does not exist.  Stop.
Device j3xlte not found. Attempting to retrieve device repository from LineageOS Github (http://github.com/LineageOS).
Repository for j3xlte not found in the LineageOS Github repository list. If this is in error, you may need to manually add it to your local_manifests/roomservice.xml.
build/core/product_config.mk:254: *** _nic.PRODUCTS.[[device/samsung/j3xlte/lineage.mk]]: "device/samsung/sharkls-common/sharkls.mk" does not exist.  Stop.
build/core/product_config.mk:254: *** _nic.PRODUCTS.[[device/samsung/j3xlte/lineage.mk]]: "device/samsung/sharkls-common/sharkls.mk" does not exist.  Stop.

** Don't have a product spec for: 'lineage_j3xlte'
** Do you have the right repo manifest?

It looks like the device/samsung/sharkls-common was not checked out properly. After re-doing with force-sync, it failed with other errors:

$ breakfast j3xlte
including vendor/cm/vendorsetup.sh
build/core/product_config.mk:254: *** _nic.PRODUCTS.[[device/samsung/j3xlte/lineage.mk]]: "device/common/gps/gps_eu_supl.mk" does not exist.  Stop.
build/core/product_config.mk:254: *** _nic.PRODUCTS.[[device/samsung/j3xlte/lineage.mk]]: "device/common/gps/gps_eu_supl.mk" does not exist.  Stop.
build/core/product_config.mk:254: *** _nic.PRODUCTS.[[device/samsung/j3xlte/lineage.mk]]: "device/common/gps/gps_eu_supl.mk" does not exist.  Stop.
Device j3xlte not found. Attempting to retrieve device repository from LineageOS Github (http://github.com/LineageOS).
Repository for j3xlte not found in the LineageOS Github repository list. If this is in error, you may need to manually add it to your local_manifests/roomservice.xml.
build/core/product_config.mk:254: *** _nic.PRODUCTS.[[device/samsung/j3xlte/lineage.mk]]: "device/common/gps/gps_eu_supl.mk" does not exist.  Stop.
build/core/product_config.mk:254: *** _nic.PRODUCTS.[[device/samsung/j3xlte/lineage.mk]]: "device/common/gps/gps_eu_supl.mk" does not exist.  Stop.

** Don't have a product spec for: 'lineage_j3xlte'
** Do you have the right repo manifest?

Indeed, I had nothing under device/common. So, I looked under halium/devices/manifests to see if any other manifest was populating this directory, and found that the bq_krillin.xml file had such a line. So, I copied that line (the one adding the lineageos/android_device_common project) into my manifest. After doing that, the breakfast command succeeded:

$ breakfast j3xlte
including vendor/cm/vendorsetup.sh
Trying dependencies-only mode on a non-existing device tree?

============================================
PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=7.1.1
LINEAGE_VERSION=14.1-20190619-UNOFFICIAL-j3xlte
TARGET_PRODUCT=lineage_j3xlte
TARGET_BUILD_VARIANT=userdebug
TARGET_BUILD_TYPE=release
TARGET_BUILD_APPS=
TARGET_ARCH=arm
TARGET_ARCH_VARIANT=armv7-a-neon
TARGET_CPU_VARIANT=cortex-a7
TARGET_2ND_ARCH=
TARGET_2ND_ARCH_VARIANT=
TARGET_2ND_CPU_VARIANT=
HOST_ARCH=x86_64
HOST_2ND_ARCH=x86
HOST_OS=linux
HOST_OS_EXTRA=Linux-4.15.0-51-generic-x86_64-with-Ubuntu-16.04-xenial
HOST_CROSS_OS=windows
HOST_CROSS_ARCH=x86
HOST_CROSS_2ND_ARCH=x86_64
HOST_BUILD_TYPE=release
BUILD_ID=NOF27B
OUT_DIR=/mnt/Lavoro/halium/out
RECOVERY_VARIANT=twrp
WITH_SU=true
============================================

Building the kernel

According to the livestream, the proper script to check for the kernel config is not the Mer one, but halium/halium-boot/check-kernel-config (this is also documented in the Ubports - Building Halium boot documentation page).

At this point I realized that I didn't have a kernel: an entry starting with kernel/ should always be in the manifest file. So I added the kernel line taken from the LineageOS local manifest file.

I then looked for the config file, which I eventually found in arch/arm/configs/j3xlte_defconfig. So I ran

./halium/halium-boot/check-kernel-config kernel/samsung/sharkls/arch/arm/configs/j3xlte_defconfig -w

which reported no errors. However, in the command output I noticed this:

 CONFIG_TMPFS_POSIX_ACL is already set
 CONFIG_DEFAULT_SECURITY is set, but to y "selinux" y not "apparmor".
 Setting CONFIG_DEFAULT_SECURITY="apparmor" correctly
sed: -e expression #1, char 60: unterminated `s' command
 Setting CONFIG_SECURITY_APPARMOR_BOOTPARAM_VALUE=1

When I checked the config file, I noticed that AppArmor was not really enabled. So I changed it manually.

Next, the fstab. I had the Ubports and Halium guides both opened in my browser, and I went for the "Fix mounts" step from the Ubports guide. I used find to locate the fstab, and found two:

The recovery file seemed already fine to me, while the rootdir one had most entry duplicated; ones with f2fs and xattrs, the other ones with ext4. I commented out the f2fs ones and ran mka halium-boot:

[...]
/mnt/Lavoro/halium/out/build-aosp_arm.ninja is missing, regenerating...
============================================
PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=7.1.1
LINEAGE_VERSION=
TARGET_PRODUCT=aosp_arm
TARGET_BUILD_VARIANT=eng
TARGET_BUILD_TYPE=release
TARGET_BUILD_APPS=
TARGET_ARCH=arm
TARGET_ARCH_VARIANT=armv7-a
TARGET_CPU_VARIANT=generic
TARGET_2ND_ARCH=
TARGET_2ND_ARCH_VARIANT=
TARGET_2ND_CPU_VARIANT=
HOST_ARCH=x86_64
HOST_2ND_ARCH=x86
HOST_OS=linux
HOST_OS_EXTRA=Linux-4.15.0-51-generic-x86_64-with-Ubuntu-16.04-xenial
HOST_CROSS_OS=windows
HOST_CROSS_ARCH=x86
HOST_CROSS_2ND_ARCH=x86_64
HOST_BUILD_TYPE=release
BUILD_ID=NOF27B
OUT_DIR=/mnt/Lavoro/halium/out
============================================
Checking build tools versions...
find: ‘device/*/generic’: No such file or directory
find: ‘device/unknown’: No such file or directory
find: ‘device/android’: No such file or directory
find: ‘device/*/generic’: No such file or directory
find: ‘device/unknown’: No such file or directory
find: ‘device/android’: No such file or directory
halium/hybris-boot/Android.mk:69: ********************* /boot appears to live on ERROR: *fstab* not found
halium/hybris-boot/Android.mk:70: ********************* /data appears to live on ERROR: *fstab* not found
halium/hybris-boot/Android.mk:73: *** There should be a one and only one device entry for HYBRIS_BOOT_PART and HYBRIS_DATA_PART.
build/core/ninja.mk:166: recipe for target '/mnt/Lavoro/halium/out/build-aosp_arm.ninja' failed
make: *** [/mnt/Lavoro/halium/out/build-aosp_arm.ninja] Error 1
make: Leaving directory '/mnt/Lavoro/halium'

#### make failed to build some targets (11 seconds) ####

this seems to be documented in the "Common system build errors" page: I had to rerun source build/envsetup.sh and breakfast j3xlte. After that, the mka halium-boot command failed with

device/samsung/sharkls-common/ims/sec_samsung/Android.mk:25: build/core/java_library.mk: No such file or directory
build/core/ninja.mk:166: recipe for target '/mnt/Lavoro/halium/out/build-lineage_j3xlte.ninja' failed
make: *** [/mnt/Lavoro/halium/out/build-lineage_j3xlte.ninja] Error 1
make: Leaving directory '/mnt/Lavoro/halium'

which looks like a progress. The issue about the missing java_library.mk has been raised in the #halium channel; possible solutions include:

  • symlink from static_java_library.mk
  • edit the Android.mk file

I went for the former; this probably needs to be fixed in Halium.

Reran mka halium-boot, got some improvement: fstab is found:

halium/hybris-boot/Android.mk:70: ********************* /boot appears to live on 
halium/hybris-boot/Android.mk:71: ********************* /data appears to live on /dev/block/platform/sdio_emmc/by-name/userdata
halium/hybris-boot/Android.mk:74: *** There should be a one and only one device entry for HYBRIS_BOOT_PART and HYBRIS_DATA_PART.
build/core/ninja.mk:166: recipe for target '/mnt/Lavoro/halium/out/build-lineage_j3xlte.ninja' failed
make: *** [/mnt/Lavoro/halium/out/build-lineage_j3xlte.ninja] Error 1
make: Leaving directory '/mnt/Lavoro/halium'

It looks like my fstab was not complete, but missing the /boot partition. given that none of the fstab I located in my source tree had a line for the boot partition, I checked out the Android's fstab from the phone, via adb shell. There is no fstab in /etc, but two fstab files right in the root directory of the filesystem: fstab.sc8830 with exactly the same contents as the recovery fstab I found before in device/samsung/sharkls-common/recovery/root/fstab.sc8830, and a fstab.goldfish:

127|root@j3xlte:/ # cat /fstab.goldfish                                        
# Android fstab file.
#<src>                                                  <mnt_point>         <type>    <mnt_flags and options>                              <fs_mgr_flags>
# The filesystem that contains the filesystem checker binary (typically /system) cannot
# specify MF_CHECK, and must come before any filesystems that do specify MF_CHECK
/dev/block/mtdblock0                                    /system             ext4      ro,noatime,barrier=1                                         wait
/dev/block/mtdblock1                                    /data               ext4      noatime,nosuid,nodev,barrier=1,nomblk_io_submit      wait,check
/dev/block/mtdblock2                                    /cache              ext4      noatime,nosuid,nodev  wait,check
/devices/platform/goldfish_mmc.0                        auto                vfat      defaults                                             voldmanaged=sdcard:auto

While mount says:

~ # mount
rootfs on / type rootfs (rw,seclabel)
tmpfs on /dev type tmpfs (rw,seclabel,nosuid,relatime,mode=755)
devpts on /dev/pts type devpts (rw,seclabel,relatime,mode=600)
proc on /proc type proc (rw,relatime,gid=3009,hidepid=2)
sysfs on /sys type sysfs (rw,seclabel,relatime)
selinuxfs on /sys/fs/selinux type selinuxfs (rw,relatime)
tmpfs on /tmp type tmpfs (rw,seclabel,relatime)
pstore on /sys/fs/pstore type pstore (rw,seclabel,relatime)
adb on /dev/usb-ffs/adb type functionfs (rw,relatime)
adb on /dev/usb-ffs/adb type functionfs (rw,relatime)
/dev/block/mmcblk0p27 on /data type ext4 (rw,seclabel,relatime,data=ordered)
/dev/block/mmcblk0p27 on /sdcard type ext4 (rw,seclabel,relatime,data=ordered)
/dev/block/mmcblk0p24 on /cache type ext4 (rw,seclabel,relatime,data=ordered)

None of the above helps in locating the boot partition. I then tried to look under /dev/block:

root@j3xlte:/dev/block/platform/sdio_emmc/by-name # ls -l
lrwxrwxrwx root     root              2019-06-26 19:13 CACHE -> /dev/block/mmcblk0p24
lrwxrwxrwx root     root              2019-06-26 19:13 FOTA_SIG -> /dev/block/mmcblk0p11
lrwxrwxrwx root     root              2019-06-26 19:13 HIDDEN -> /dev/block/mmcblk0p26
lrwxrwxrwx root     root              2019-06-26 19:13 KERNEL -> /dev/block/mmcblk0p20
lrwxrwxrwx root     root              2019-06-26 19:13 PARAM -> /dev/block/mmcblk0p16
lrwxrwxrwx root     root              2019-06-26 19:13 PERSDATA -> /dev/block/mmcblk0p23
lrwxrwxrwx root     root              2019-06-26 19:13 PERSISTENT -> /dev/block/mmcblk0p22
lrwxrwxrwx root     root              2019-06-26 19:13 RECOVERY -> /dev/block/mmcblk0p21
lrwxrwxrwx root     root              2019-06-26 19:13 RESERVED2 -> /dev/block/mmcblk0p19
lrwxrwxrwx root     root              2019-06-26 19:13 SBOOT -> /dev/block/mmcblk0p1
lrwxrwxrwx root     root              2019-06-26 19:13 SBOOT2 -> /dev/block/mmcblk0p2
lrwxrwxrwx root     root              2019-06-26 19:13 SYSTEM -> /dev/block/mmcblk0p25
lrwxrwxrwx root     root              2019-06-26 19:13 efs -> /dev/block/mmcblk0p17
lrwxrwxrwx root     root              2019-06-26 19:13 l_fixnv1 -> /dev/block/mmcblk0p3
lrwxrwxrwx root     root              2019-06-26 19:13 l_fixnv2 -> /dev/block/mmcblk0p4
lrwxrwxrwx root     root              2019-06-26 19:13 l_gdsp -> /dev/block/mmcblk0p9
lrwxrwxrwx root     root              2019-06-26 19:13 l_ldsp -> /dev/block/mmcblk0p7
lrwxrwxrwx root     root              2019-06-26 19:13 l_modem -> /dev/block/mmcblk0p8
lrwxrwxrwx root     root              2019-06-26 19:13 l_runtimenv1 -> /dev/block/mmcblk0p12
lrwxrwxrwx root     root              2019-06-26 19:13 l_runtimenv2 -> /dev/block/mmcblk0p13
lrwxrwxrwx root     root              2019-06-26 19:13 l_warm -> /dev/block/mmcblk0p10
lrwxrwxrwx root     root              2019-06-26 19:13 pm_sys -> /dev/block/mmcblk0p5
lrwxrwxrwx root     root              2019-06-26 19:13 prodnv -> /dev/block/mmcblk0p18
lrwxrwxrwx root     root              2019-06-26 19:13 rsvdfixnv1 -> /dev/block/mmcblk0p6
lrwxrwxrwx root     root              2019-06-26 19:13 td_runtimenv1 -> /dev/block/mmcblk0p14
lrwxrwxrwx root     root              2019-06-26 19:13 td_runtimenv2 -> /dev/block/mmcblk0p15
lrwxrwxrwx root     root              2019-06-26 19:13 userdata -> /dev/block/mmcblk0p27

This, combined from the PIT file which we can extract with Heimdall from the "Odin mode", gives us a better understanding of the device partitions:

$ sudo heimdall print-pit --no-reboot
Heimdall v1.4.2

Copyright (c) 2010-2017 Benjamin Dobell, Glass Echidna
http://www.glassechidna.com.au/

This software is provided free of charge. Copying and redistribution is
encouraged.

If you appreciate this software and you would like to support future
development please consider donating:
http://www.glassechidna.com.au/donate/

Initialising connection...
Detecting device...
Claiming interface...
Setting up interface...

Initialising protocol...
Protocol initialisation successful.

Beginning session...

Some devices may take up to 2 minutes to respond.
Please be patient!

Session begun.

Downloading device's PIT file...
PIT file download successful.

Entry Count: 31
Unknown 1: 1598902083
Unknown 2: 844251476
Unknown 3: 20563
Unknown 4: 17490
Unknown 5: 14136
Unknown 6: 13619
Unknown 7: 0
Unknown 8: 0


--- Entry #0 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 80
Attributes: 2 (STL Read-Only)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 0
Partition Block Count: 1024
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: BOOT
Flash Filename: spl.img
FOTA Filename: 


--- Entry #1 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 90
Attributes: 2 (STL Read-Only)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 0
Partition Block Count: 2048
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: BOOT2
Flash Filename: spl2.img
FOTA Filename: 


--- Entry #2 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 70
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 1024
Partition Block Count: 1024
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: PIT
Flash Filename: J3XLTE.pit
FOTA Filename: 


--- Entry #3 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 71
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 2048
Partition Block Count: 6144
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: MD5HDR
Flash Filename: md5.img
FOTA Filename: 


--- Entry #4 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 1
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 8192
Partition Block Count: 4096
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: SBOOT
Flash Filename: sboot.bin
FOTA Filename: 


--- Entry #5 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 2
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 12288
Partition Block Count: 4096
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: SBOOT2
Flash Filename: sboot2.bin
FOTA Filename: 


--- Entry #6 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 3
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 16384
Partition Block Count: 2048
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: l_fixnv1
Flash Filename: nvitem1.bin
FOTA Filename: 


--- Entry #7 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 4
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 18432
Partition Block Count: 2048
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: l_fixnv2
Flash Filename: nvitem.bin
FOTA Filename: 


--- Entry #8 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 5
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 20480
Partition Block Count: 2048
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: pm_sys
Flash Filename: PM_sharkl_arm7.bin
FOTA Filename: 


--- Entry #9 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 6
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 22528
Partition Block Count: 2048
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: rsvdfixnv1
Flash Filename: 
FOTA Filename: 


--- Entry #10 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 7
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 24576
Partition Block Count: 8192
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: l_ldsp
Flash Filename: SPRDLTEDSP.img
FOTA Filename: 


--- Entry #11 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 8
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 32768
Partition Block Count: 32768
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: l_modem
Flash Filename: SPRDCP.img
FOTA Filename: 


--- Entry #12 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 9
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 65536
Partition Block Count: 8192
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: l_gdsp
Flash Filename: SPRDGDSP.img
FOTA Filename: 


--- Entry #13 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 10
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 73728
Partition Block Count: 8192
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: l_warm
Flash Filename: SPRDWDSP.img
FOTA Filename: 


--- Entry #14 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 11
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 81920
Partition Block Count: 2048
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: FOTA_SIG
Flash Filename: 
FOTA Filename: 


--- Entry #15 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 12
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 83968
Partition Block Count: 2048
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: l_runtimenv1
Flash Filename: 
FOTA Filename: 


--- Entry #16 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 13
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 86016
Partition Block Count: 2048
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: l_runtimenv2
Flash Filename: 
FOTA Filename: 


--- Entry #17 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 14
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 88064
Partition Block Count: 2048
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: td_runtimenv1
Flash Filename: 
FOTA Filename: 


--- Entry #18 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 15
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 90112
Partition Block Count: 2048
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: td_runtimenv2
Flash Filename: 
FOTA Filename: 


--- Entry #19 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 16
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 92160
Partition Block Count: 4096
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: PARAM
Flash Filename: param.lfs
FOTA Filename: 


--- Entry #20 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 17
Attributes: 5 (Read/Write)
Update Attributes: 5 (FOTA)
Partition Block Size/Offset: 96256
Partition Block Count: 40960
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: efs
Flash Filename: efs.img
FOTA Filename: 


--- Entry #21 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 18
Attributes: 5 (Read/Write)
Update Attributes: 5 (FOTA)
Partition Block Size/Offset: 137216
Partition Block Count: 10240
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: prodnv
Flash Filename: prodnv.img
FOTA Filename: 


--- Entry #22 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 19
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 147456
Partition Block Count: 12288
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: RESERVED2
Flash Filename: 
FOTA Filename: 


--- Entry #23 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 20
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 159744
Partition Block Count: 40960
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: KERNEL
Flash Filename: boot.img
FOTA Filename: 


--- Entry #24 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 21
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 200704
Partition Block Count: 40960
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: RECOVERY
Flash Filename: recovery.img
FOTA Filename: 


--- Entry #25 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 22
Attributes: 5 (Read/Write)
Update Attributes: 1 (FOTA)
Partition Block Size/Offset: 241664
Partition Block Count: 2048
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: PERSISTENT
Flash Filename: 
FOTA Filename: 


--- Entry #26 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 23
Attributes: 5 (Read/Write)
Update Attributes: 5 (FOTA)
Partition Block Size/Offset: 243712
Partition Block Count: 18432
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: PERSDATA
Flash Filename: persdata.img
FOTA Filename: 


--- Entry #27 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 24
Attributes: 5 (Read/Write)
Update Attributes: 5 (FOTA)
Partition Block Size/Offset: 262144
Partition Block Count: 409600
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: CACHE
Flash Filename: cache.img
FOTA Filename: 


--- Entry #28 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 25
Attributes: 5 (Read/Write)
Update Attributes: 5 (FOTA)
Partition Block Size/Offset: 671744
Partition Block Count: 4194304
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: SYSTEM
Flash Filename: system.img
FOTA Filename: 


--- Entry #29 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 26
Attributes: 5 (Read/Write)
Update Attributes: 5 (FOTA)
Partition Block Size/Offset: 4866048
Partition Block Count: 81920
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: HIDDEN
Flash Filename: hidden.img
FOTA Filename: 


--- Entry #30 ---
Binary Type: 0 (AP)
Device Type: 2 (MMC)
Identifier: 27
Attributes: 5 (Read/Write)
Update Attributes: 5 (FOTA)
Partition Block Size/Offset: 4947968
Partition Block Count: 0
File Offset (Obsolete): 0
File Size (Obsolete): 0
Partition Name: userdata
Flash Filename: userdata.img
FOTA Filename: remained

Ending session...
Releasing device interface...

Looking at the name, size and flash filename of the partitions, we can guess that the BOOT and BOOT2 are about splash screens, and what we need to use as our boot partition is in fact the KERNEL partition: /dev/block/mmcblk0p20 (note how the Identifier field from the PIT file matches what we got from block devices in the /dev filesystem). So, we edit our fstab file and add a line

/dev/block/platform/sdio_emmc/by-name/KERNEL   /boot    ext4 ro     wait

From the IRC #halium channel, I understood that the fixup mountpoints step is needed. The block I added to the halium/hybris-boot/fixup-mountpoints script is this:

    "j3xlte")
        sed -i \
            -e 's block/platform/sdio_emmc/by-name/SYSTEM mmcblk0p25 ' \
            -e 's block/platform/sdio_emmc/by-name/userdata mmcblk0p27 ' \
            -e 's block/platform/sdio_emmc/by-name/CACHE mmcblk0p24 ' \
            -e 's block/platform/sdio_emmc/by-name/efs mmcblk0p17 ' \
            -e 's block/platform/sdio_emmc/by-name/prodnv mmcblk0p18 ' \
            -e 's block/platform/sdio_emmc/by-name/KERNEL mmcblk0p20 ' \
            "$@"
        ;;

I then ran mka halium-boot again, and it finally started to build the kernel. But it soon stopped:

/mnt/Lavoro/halium/kernel/samsung/sharkls/kernel/cgroup.c: In function 'subsys_cgroup_allow_attach':
/mnt/Lavoro/halium/kernel/samsung/sharkls/kernel/cgroup.c:2138:37: error: invalid operands to binary != (have 'kuid_t' and 'kuid_t')
   if (current != task && cred->euid != tcred->uid &&
                                     ^
/mnt/Lavoro/halium/kernel/samsung/sharkls/kernel/cgroup.c:2139:18: error: invalid operands to binary != (have 'kuid_t' and 'kuid_t')
       cred->euid != tcred->suid)
                  ^

Luckily, this is one of the known build errors, so I didn't waste a lot of time on it: setting CONFIG_USER_NS=n in kernel/samsung/sharkls/arch/arm/configs/j3xlte_defconfig fixed it. Then the build failed with

  LD      init/built-in.o
mm/built-in.o: In function `set_AKSM_level':
/mnt/Lavoro/halium/kernel/samsung/sharkls/mm/ksm.c:1822: undefined reference to `get_minfree_high_value'
/mnt/Lavoro/halium/kernel/samsung/sharkls/Makefile:786: recipe for target 'vmlinux' failed
make[1]: *** [vmlinux] Error 1
Makefile:130: recipe for target 'sub-make' failed
make: *** [sub-make] Error 2

Looking at the code, I understood I could probably switch off this code path with CONFIG_ADAPTIVE_KSM=n. After doing that, got another error:

  LD      init/built-in.o
arch/arm/lib/lib.a(memcmpksm.o):(.ARM.exidx+0x0): undefined reference to `__aeabi_unwind_cpp_pr0'
/mnt/Lavoro/halium/kernel/samsung/sharkls/Makefile:786: recipe for target 'vmlinux' failed
make[1]: *** [vmlinux] Error 1
Makefile:130: recipe for target 'sub-make' failed
make: *** [sub-make] Error 2

Disabling also CONFIG_KSM_ASSEMBLY_MEMCMP did the trick:

  CC      /mnt/Lavoro/halium/vendor/sprd/wcn/wifi/sc2331/6.0/sprdwl.mod.o
  LD [M]  /mnt/Lavoro/halium/vendor/sprd/wcn/wifi/sc2331/6.0/sprdwl.ko
make[1]: Leaving directory '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/KERNEL_OBJ'
make: Leaving directory '/mnt/Lavoro/halium/vendor/sprd/wcn/wifi/sc2331/6.0'
[ 77% 7/9] Target dt image: /mnt/Lavoro/halium/out/target/product/j3xlte/dt.img
DTB combiner:
  Input directory: '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/KERNEL_OBJ/arch/arm/boot/dts/'
  Output file: '/mnt/Lavoro/halium/out/target/product/j3xlte/dt.img'
Found file: sprd-scx35l_sharkls_j3xlte_rev03.dtb ... chipset: 9830, platform: 3, rev: 131072
Found file: sprd-scx35l_sharkls_j3xlte_rev02.dtb ... chipset: 9830, platform: 2, rev: 131072
Found file: sprd-scx35l_sharkls_j3xlte_rev06.dtb ... chipset: 9830, platform: 6, rev: 131072
Found file: sprd-scx35l_sharkls_j3xlte_rev07.dtb ... chipset: 9830, platform: 7, rev: 131072
Found file: sprd-scx35l_sharkls_j3xlte_rev04.dtb ... chipset: 9830, platform: 4, rev: 131072
Found file: sprd-scx35l_sharkls_j3xlte_rev01.dtb ... chipset: 9830, platform: 1, rev: 131072
Found file: sprd-scx35l_sharkls_j3xlte_rev05.dtb ... chipset: 9830, platform: 5, rev: 131072
Found file: sprd-scx35l_sharkls_j3xlte_rev00.dtb ... chipset: 9830, platform: 0, rev: 131072
=> Found 8 unique DTB(s)

Generating master DTB... completed
Made DT image: /mnt/Lavoro/halium/out/target/product/j3xlte/dt.img
[100% 9/9] Install: /mnt/Lavoro/halium/out/target/product/j3xlte/halium-boot.img
make: Leaving directory '/mnt/Lavoro/halium'

#### make completed successfully (01:28 (mm:ss)) ####

Building the Halium system image

Continuing following the Halium documentation, the next step was building the system image with the mka systemimage command. Of course, it failed:

make: Entering directory '/mnt/Lavoro/halium'
============================================
PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=7.1.1
LINEAGE_VERSION=14.1-20190702-UNOFFICIAL-j3xlte
TARGET_PRODUCT=lineage_j3xlte
TARGET_BUILD_VARIANT=userdebug
TARGET_BUILD_TYPE=release
TARGET_BUILD_APPS=
TARGET_ARCH=arm
TARGET_ARCH_VARIANT=armv7-a-neon
TARGET_CPU_VARIANT=cortex-a7
TARGET_2ND_ARCH=
TARGET_2ND_ARCH_VARIANT=
TARGET_2ND_CPU_VARIANT=
HOST_ARCH=x86_64
HOST_2ND_ARCH=x86
HOST_OS=linux
HOST_OS_EXTRA=Linux-4.15.0-52-generic-x86_64-with-Ubuntu-16.04-xenial
HOST_CROSS_OS=windows
HOST_CROSS_ARCH=x86
HOST_CROSS_2ND_ARCH=x86_64
HOST_BUILD_TYPE=release
BUILD_ID=NOF27B
OUT_DIR=/mnt/Lavoro/halium/out
RECOVERY_VARIANT=twrp
WITH_SU=true
============================================
Running kati to generate build-lineage_j3xlte.ninja...
No need to regenerate ninja file
Starting build with ninja
ninja: Entering directory `.'
ninja: error: '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libcurl_intermediates/export_includes', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libstrongswan_intermediates/import_includes', missing and no known rule to make it
build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed
make: *** [ninja_wrapper] Error 1
make: Leaving directory '/mnt/Lavoro/halium'

#### make failed to build some targets (2 seconds) ####

I found similar (though not exactly the same) errors in #halium: the solution is to locate the .mk file causing the build of the package requiring the missing file, and remove it. I ran a repo grep libstrongswan, and found that it's mentioned in device/samsung/sharkls-common/sharkls.mk. I removed it from there, but I got the same build error. Removing the whole block of PRODUCT_PACKAGES led to some progress:

build/core/binary.mk:1253: vendor/sprd/modules/libcamera/Android.mk: libcamisp2.0: Unused source files: isp2.0/calibration/backup isp2.0/third_lib/alc_ip
build/core/binary.mk:1253: vendor/sprd/open-source/libs/vpu/mmf/openmax/libomxil-bellagio-0.9.3/Android.mk: omxregister-bellagio: Unused source files: src/omxregister.h
No private recovery resources for TARGET_DEVICE j3xlte
build/core/Makefile:34: warning: overriding commands for target `/mnt/Lavoro/halium/out/target/product/j3xlte/system/etc/wifi/wpa_supplicant.conf'
build/core/base_rules.mk:316: warning: ignoring old commands for target `/mnt/Lavoro/halium/out/target/product/j3xlte/system/etc/wifi/wpa_supplicant.conf'
Starting build with ninja
ninja: Entering directory `.'
ninja: error: '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/STATIC_LIBRARIES/libbt-utils_intermediates/export_includes', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/EXECUTABLES/engpc_intermediates/import_includes', missing and no known rule to make it
build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed
make: *** [ninja_wrapper] Error 1

I then removed engpc from the same sharkls.mk file:

Starting build with ninja
ninja: Entering directory `.'
ninja: error: '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/STATIC_LIBRARIES/libbootloader_message_intermediates/export_includes', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/EXECUTABLES/vold_intermediates/import_includes', missing and no known rule to make it
build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed
make: *** [ninja_wrapper] Error 1

The last comment from here suggests that vold might not be needed (though some people in #halium claimed that it could be needed, and indeed in the Meizu Pro 5 by Canonical I do have that program under /system/bin/). Anyway, I edited build/target/product/base.mk and removed vold from the PRODUCT_PACKAGES variable. I also read that it's recommended to run the make command as LANG=C mka systemimage, since the outcome can be affected by the user locale. Next:

Starting build with ninja
ninja: Entering directory `.'
ninja: error: '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libfmjni_intermediates/export_includes', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengfm_intermediates/import_includes', missing and no known rule to make it
build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed
make: *** [ninja_wrapper] Error 1

This is mentioned in vendor/sprd/wcn/platform/wcnd/Android.mk, so let's remove it from there (LOCAL_SHARED_LIBRARIES variable):

Starting build with ninja
ninja: Entering directory `.'
ninja: error: 'vendor/samsung/j3xlte/proprietary/bin/at_distributor', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/system/bin/at_distributor', missing and no known rule to make it
build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed
make: *** [ninja_wrapper] Error 1

This is brought in by device/samsung/j3xlte/proprietary-files.txt, so I commented it out from there. This didn't seem to make any difference, and a mka clean didn't help either. Turns out it's also mentioned in vendor/samsung/j3xlte/j3xlte-vendor-blobs.mk, so I removed it from there too (PRODUCT_COPY_FILES variable). Reading the comment block on top of this makefile we understand that it's generated by the setup-makefiles.sh script based on the contents of proprietary-files.txt. So, one option is to re-run the setup-makefiles.sh script, but I preferred to take a no risk approach and edit both files by hand. The next failure was about mfgloader, which is also mentioned in the same file:

Starting build with ninja
ninja: Entering directory `.'
ninja: error: 'vendor/samsung/j3xlte/proprietary/bin/mfgloader', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/system/bin/mfgloader', missing and no known rule to make it
build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed
make: *** [ninja_wrapper] Error 1

Let's try to remove it as well, from the same two files:

Starting build with ninja
ninja: Entering directory `.'
ninja: error: 'vendor/samsung/j3xlte/proprietary/bin/sprdSleepLog', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/system/bin/sprdSleepLog', missing and no known rule to make it
build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed
make: *** [ninja_wrapper] Error 1

Next is modemd (exactly same logs). At this point I started suspecting that this was not the right way to proceed, that that instead I was just missing all the proprietary blobs. The device tree repository has an extract-files.sh which seems to get these binaries from a device. After looking at its source, it looks like it must be run from the device/samsung/j3xlte directory with the device connected and reachable via adb. Before running it, I reverted my changes to the proprietary-files.txt file, in order to retrieve all the needed blobs. Running the extraction tool failed once:

./extract-files.sh 
846 KB/s (92260 bytes in 0.106s)
108 KB/s (9664 bytes in 0.086s)
106 KB/s (9444 bytes in 0.086s)
279 KB/s (34092 bytes in 0.119s)
44 KB/s (1992 bytes in 0.043s)
115 KB/s (5296 bytes in 0.044s)
2 KB/s (265 bytes in 0.087s)
40 KB/s (3770 bytes in 0.090s)
25 KB/s (2247 bytes in 0.087s)
4 KB/s (427 bytes in 0.087s)
0 KB/s (70 bytes in 0.087s)
remote object '/system/bin/IPSecService' does not exist

but after removing the missing file from proprietary-files.txt, the extraction completed successfully. However, one file was still not retrieved, as this line from the output showed:

[...]
4779 KB/s (1386252 bytes in 0.283s)
405 KB/s (37928 bytes in 0.091s)
failed to copy '/system/vendor/firmware/vbc_eq' to '../../../vendor/samsung/j3xlte/proprietary/vendor/firmware/vbc_eq': Permission denied
3015 KB/s (415875 bytes in 0.134s)
[...]

I checked the situation by connecting to the device with adb shell, and the file was there but readable by root only. After copying the file to /data/user and calling chmod 0666 vbc_eq, I could manually retrieve it with

adb pull /data/user/vbc_eq ../../../vendor/samsung/j3xlte/proprietary/vendor/firmware/vbc_eq

I preferred not to touch the permissions directly on the original file, not to risk messing up with the system; and after retrieving the file, I removed my copy from the device. The I ran

JOBS=2 ./halium/devices/setup j3xlte

once again. That completed fine, and the next mka systemimage failed with another error:

Starting build with ninja
ninja: Entering directory `.'
ninja: error: '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/STATIC_LIBRARIES/libbootloader_message_intermediates/export_includes', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/EXECUTABLES/init_intermediates/import_includes', missing and no known rule to make it
build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed
make: *** [ninja_wrapper] Error 1

I played with the android_system_core repository, which is checked out in system/core/, to remove references to bootloader_message from init/builtins.cpp, and I also removed it from the LOCAL_STATIC_LIBRARIES variable in the system/core/init/Android.mk file. Next:

Starting build with ninja
ninja: Entering directory `.'
ninja: error: '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/STATIC_LIBRARIES/libminui_intermediates/export_includes', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/EXECUTABLES/healthd_intermediates/import_includes', missing and no known rule to make it
build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed
make: *** [ninja_wrapper] Error 1

This is listed in build/target/product/core_base.mk and build/target/product/embedded.mk and needs to be removed from both (it was disabled in Halium's init, so it's definitely not needed). It then failed with:

Starting build with ninja
ninja: Entering directory `.'
ninja: error: 'bootable/recovery-twrp/etc/init.rc', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/ramdisk-recovery.cpio', missing and no known rule to make it
build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed
make: *** [ninja_wrapper] Error 1

The recovery-twrp project is listed in device/samsung/sharkls-common/lineage.dependencies, so let's try adding a line

<project path="bootable/recovery-twrp" name="omnirom/android_bootable_recovery" remote="github" revision="android-7.1" />

to our manifest file and run JOBS=2 ./halium/devices/setup j3xlte once more. After that, mka systemimage gives another error:

Starting build with ninja
ninja: Entering directory `.'
ninja: error: '/mnt/Lavoro/halium/out/host/linux-x86/bin/imgdiff', needed by '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/PACKAGING/recovery_patch_intermediates/recovery_from_boot.p', missing and no known rule to make it
build/core/ninja.mk:151: recipe for target 'ninja_wrapper' failed
make: *** [ninja_wrapper] Error 1

Adding bootable/recovery-twrp to the subdirs variable in build/core/main.mk seems to get us further. But before showing you the next error, I need to stop for a second, because at this point I started suspecting that the previous error about the bootloader_message library could go away now that I started using the same recovery-twrp tree used by the LineageOS porter. So I reverted my changes about the bootloader_message library and indeed it looked like the previous error was gone forever. And it might be that some of the other stuff I disabled above might be restored back; but I decided to leave it for later (it's documented in this blog post, after all, so I can go back to it at any time) and proceed with the build:

Starting build with ninja
ninja: Entering directory `.'
[  0% 125/16364] host C++: libadb <= system/core/adb/usb_linux.cpp
FAILED: /bin/bash -c "(PWD=/proc/self/cwd  prebuilts/clang/host/linux-x86/clang-2690385/bin/clang++ -I device/samsung/sharkls-common/include -I system/core/adb -I /mnt/Lavoro/halium/out/host/linux-x86/obj/STATIC_LIBRARIES/libadb_intermediates -I /mnt/Lavoro/halium/out/host/linux-x86/gen/STATIC_LIBRARIES/libadb_intermediates -I libnativehelper/include/nativehelper \$(cat /mnt/Lavoro/halium/out/host/linux-x86/obj/STATIC_LIBRARIES/libadb_intermediates/import_includes) -isystem system/core/include -isystem system/media/audio/include -isystem hardware/libhardware/include -isystem hardware/libhardware_legacy/include -isystem libnativehelper/include -isystem frameworks/native/include -isystem frameworks/native/opengl/include -isystem frameworks/av/include -isystem frameworks/base/include -isystem tools/include -isystem /mnt/Lavoro/halium/out/host/linux-x86/obj/include -c    -fno-exceptions -Wno-multichar -m64 -Wa,--noexecstack -fPIC -no-canonical-prefixes -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector -D__STDC_FORMAT_MACROS -D__STDC_CONSTANT_MACROS -DANDROID -fmessage-length=0 -W -Wall -Wno-unused -Winit-self -Wpointer-arith -O2 -g -fno-strict-aliasing -DNDEBUG -UDEBUG  -D__compiler_offsetof=__builtin_offsetof -Werror=int-conversion -Wno-reserved-id-macro -Wno-format-pedantic -Wno-unused-command-line-argument -fcolor-diagnostics   --gcc-toolchain=prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8 -fstack-protector-strong    --gcc-toolchain=prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8 --sysroot prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8/sysroot -Bprebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8/x86_64-linux/bin -target x86_64-linux-gnu   -Wsign-promo  -Wno-inconsistent-missing-override   --gcc-toolchain=prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8 --sysroot prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8/sysroot -isystem prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8/x86_64-linux/include/c++/4.8 -isystem prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8/x86_64-linux/include/c++/4.8/x86_64-linux -isystem prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8/x86_64-linux/include/c++/4.8/backward -target x86_64-linux-gnu  -Wall -Wextra -Werror -Wno-unused-parameter -Wno-missing-field-initializers -Wvla -DADB_REVISION='\"80811505f1ff-android\"' -fvisibility=hidden -DADB_HOST=1 -fPIC -std=c++14 -Wexit-time-destructors -D_USING_LIBCXX -std=gnu++14 -nostdinc++  -Werror=int-to-pointer-cast -Werror=pointer-to-int-cast  -Werror=address-of-temporary -Werror=null-dereference -Werror=return-type    -MD -MF /mnt/Lavoro/halium/out/host/linux-x86/obj/STATIC_LIBRARIES/libadb_intermediates/usb_linux.d -o /mnt/Lavoro/halium/out/host/linux-x86/obj/STATIC_LIBRARIES/libadb_intermediates/usb_linux.o system/core/adb/usb_linux.cpp ) && (cp /mnt/Lavoro/halium/out/host/linux-x86/obj/STATIC_LIBRARIES/libadb_intermediates/usb_linux.d /mnt/Lavoro/halium/out/host/linux-x86/obj/STATIC_LIBRARIES/libadb_intermediates/usb_linux.P; sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\\\\$//' -e '/^\$/ d' -e 's/\$/ :/' < /mnt/Lavoro/halium/out/host/linux-x86/obj/STATIC_LIBRARIES/libadb_intermediates/usb_linux.d >> /mnt/Lavoro/halium/out/host/linux-x86/obj/STATIC_LIBRARIES/libadb_intermediates/usb_linux.P; rm -f /mnt/Lavoro/halium/out/host/linux-x86/obj/STATIC_LIBRARIES/libadb_intermediates/usb_linux.d )"
In file included from system/core/adb/usb_linux.cpp:25:
In file included from device/samsung/sharkls-common/include/linux/usb/ch9.h:35:
In file included from device/samsung/sharkls-common/include/uapi/linux/usb/ch9.h:36:
In file included from device/samsung/sharkls-common/include/linux/types.h:5:
In file included from device/samsung/sharkls-common/include/uapi/linux/types.h:13:
In file included from prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8/sysroot/usr/include/linux/posix_types.h:4:
device/samsung/sharkls-common/include/linux/stddef.h:11:2: error: expected identifier
        false   = 0,
        ^
prebuilts/clang/host/linux-x86/clang-2690385/bin/../lib64/clang/3.8.256229/include/stdbool.h:38:15: note: expanded from macro 'false'
#define false false
              ^

(see here for the full log). After some resultless internet search and asking around, I tried to look into the issue myself, and noticed that in that failed command there's a mix of headers coming from the userspace and from the kernel (which usually are not used together). Since we are building userspace programs now, it seems a bit strange that there are also kernel headers being used (those under device/samsung/sharkls-common). Looking at system/core/adb/usb_linux.cpp, I saw that it contains a line #include <linux/usb/ch9.h> which causes the inclusion of device/samsung/sharkls-common/include/linux/usb/ch9.h; but maybe there are other ch9.h files in the build tree, provided by userspace libraries and not by the kernel? Indeed, a find showed that I had also a ./prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.15-4.8/sysroot/usr/include/linux/usb/ch9.h file, which should probably be used instead of the kernel's one. So I copied the failing command line (starting from what comes after FAILED:) into a terminal and ran it, and could reproduce the issue; but if after removing the first include directive (-I device/samsung/sharkls-common/include) the command succeeded.

The line is added in build/core/binary.mk:

my_c_includes := $(TOPDIR)$(TARGET_SPECIFIC_HEADER_PATH) $(my_c_includes)

(I found it out by modifying the file, and adding random text strings around the right-hand parts of these assignments, then running the build again and see which random strings were surrounding the device/samsung/sharkls-common/include text in the failing command line). So, we need to find why TARGET_SPECIFIC_HEADER_PATH is set. I ran a repo grep TARGET_SPECIFIC_HEADER_PATH and found that it's being set in device/samsung/sharkls-common/BoardConfigCommon.mk. Browsing the various directory trees, and double-checking the manifest file, I realized that the device/samsung/sharkls-common should not contain the kernel source tree, yet in my local disk it did (and all the files were untracked by git). It was probably the result of one of my previous attempts of compiling the manifest file, where some projects were checked out locally but I didn't clean them up after updating the manifest. I therefore ran a git clean -dfx in that directory, and the kernel source were gone. The next build continued for a while, then failed:

[ 76% 12454/16364] target thumb C: libengbt <= vendor/sprd/open-source/apps/engmode/bt/bt_cmd_executer.c
FAILED: /bin/bash -c "(PWD=/proc/self/cwd  prebuilts/clang/host/linux-x86/clang-2690385/bin/clang -I device/samsung/sharkls-common/include -I system/bt/stack/include -I system/bt/include -I vendor/sprd/open-source/apps/engmode/bt -I /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates -I /mnt/Lavoro/halium/out/target/product/j3xlte/gen/SHARED_LIBRARIES/libengbt_intermediates -I libnativehelper/include/nativehelper \$(cat /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/import_includes) -isystem system/core/include -isystem system/media/audio/include -isystem hardware/libhardware/include -isystem hardware/libhardware_legacy/include -isystem libnativehelper/include -isystem frameworks/native/include -isystem frameworks/native/opengl/include -isystem frameworks/av/include -isystem frameworks/base/include -isystem hardware/ril/include -isystem /mnt/Lavoro/halium/out/target/product/j3xlte/obj/include -isystem bionic/libc/arch-arm/include -isystem bionic/libc/include -isystem bionic/libc/kernel/uapi -isystem bionic/libc/kernel/common -isystem bionic/libc/kernel/uapi/asm-arm -isystem bionic/libm/include -isystem bionic/libm/include/arm -c    -fno-exceptions -Wno-multichar -msoft-float -ffunction-sections -fdata-sections -funwind-tables -fstack-protector-strong -Wa,--noexecstack -Werror=format-security -D_FORTIFY_SOURCE=2 -fno-short-enums -no-canonical-prefixes -mcpu=cortex-a7 -mfpu=neon-vfpv4 -D__ARM_FEATURE_LPAE=1 -mfloat-abi=softfp -DANDROID -fmessage-length=0 -W -Wall -Wno-unused -Winit-self -Wpointer-arith -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Werror=date-time -DNDEBUG -g -Wstrict-aliasing=2 -DNDEBUG -UDEBUG  -D__compiler_offsetof=__builtin_offsetof -Werror=int-conversion -Wno-reserved-id-macro -Wno-format-pedantic -Wno-unused-command-line-argument -fcolor-diagnostics -nostdlibinc  -target arm-linux-androideabi    -target arm-linux-androideabi -Bprebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/arm-linux-androideabi/bin    -std=gnu99     -mthumb -Os -fomit-frame-pointer -fno-strict-aliasing   -DHAS_NO_BDROID_BUILDCFG -DSPRD_WCNBT_MARLIN -fPIC -D_USING_LIBCXX   -Werror=int-to-pointer-cast -Werror=pointer-to-int-cast  -Werror=address-of-temporary -Werror=null-dereference -Werror=return-type  -MD -MF /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d -o /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.o vendor/sprd/open-source/apps/engmode/bt/bt_cmd_executer.c ) && (cp /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.P; sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\\\\$//' -e '/^\$/ d' -e 's/\$/ :/' < /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d >> /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.P; rm -f /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d )"
vendor/sprd/open-source/apps/engmode/bt/bt_cmd_executer.c:30:10: fatal error: 'bt_types.h' file not found
#include "bt_types.h"
         ^
1 error generated.

Indeed the bt_types.h was nowhere to be found in my source tree. A search in GitHub reveals that this file is provided by libncf-nci, so it's probably related to NFC. The local manifest from LineageOS indeed had a few lines with NFC-related projects (at least, judging from their name):

<project path="vendor/nxp-nfc/opensource/Nfc" name="LineageOS/android_vendor_nxp-nfc_opensource_Nfc" revision="cm-14.1" />
<project path="vendor/nxp-nfc/opensource/frameworks" name="LineageOS/android_vendor_nxp-nfc_opensource_frameworks" revision="cm-14.1" />
<project path="vendor/nxp-nfc/opensource/libnfc-nci" name="LineageOS/android_vendor_nxp-nfc_opensource_libnfc-nci" revision="cm-14.1" />

I initially excluded them from my manifest, given that my device doesn't have NFC, but at this point I thought I'd better include them, just for the sake of building the binaries (the alternative being removing the binaries required these libraries, or hacking on them to remove the dependency; this is a decision I might want to revisit later). So I added them to my manifest file (adding the proper remote attribute!), re-ran the setup script which fetched the newly-added repositories, but the build still failed with the same error, as the include lines were not updated. However, adding vendor/nxp-nfc/opensource/libnfc-nci/src/include to the TARGET_SPECIFIC_HEADER_PATH variable in device/samsung/sharkls-common/BoardConfigCommon.mk caused the build to fail even earlier. I then tried to add it to the local project being built (engmode), by changing the LOCAL_C_INCLUDES variable in vendor/sprd/open-source/apps/engmode/bt/Android.mk and that brought me just a little further:

[ 97% 1507/1552] target thumb C: libengbt <= vendor/sprd/open-source/apps/engmode/bt/bt_cmd_executer.c
FAILED: /bin/bash -c "(PWD=/proc/self/cwd  prebuilts/clang/host/linux-x86/clang-2690385/bin/clang -I device/samsung/sharkls-common/include -I system/bt/stack/include -I vendor/nxp-nfc/opensource/libnfc-nci/src/include -I system/bt/include -I vendor/sprd/open-source/apps/engmode/bt -I /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates -I /mnt/Lavoro/halium/out/target/product/j3xlte/gen/SHARED_LIBRARIES/libengbt_intermediates -I libnativehelper/include/nativehelper \$(cat /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/import_includes) -isystem system/core/include -isystem system/media/audio/include -isystem hardware/libhardware/include -isystem hardware/libhardware_legacy/include -isystem libnativehelper/include -isystem frameworks/native/include -isystem frameworks/native/opengl/include -isystem frameworks/av/include -isystem frameworks/base/include -isystem hardware/ril/include -isystem /mnt/Lavoro/halium/out/target/product/j3xlte/obj/include -isystem bionic/libc/arch-arm/include -isystem bionic/libc/include -isystem bionic/libc/kernel/uapi -isystem bionic/libc/kernel/common -isystem bionic/libc/kernel/uapi/asm-arm -isystem bionic/libm/include -isystem bionic/libm/include/arm -c    -fno-exceptions -Wno-multichar -msoft-float -ffunction-sections -fdata-sections -funwind-tables -fstack-protector-strong -Wa,--noexecstack -Werror=format-security -D_FORTIFY_SOURCE=2 -fno-short-enums -no-canonical-prefixes -mcpu=cortex-a7 -mfpu=neon-vfpv4 -D__ARM_FEATURE_LPAE=1 -mfloat-abi=softfp -DANDROID -fmessage-length=0 -W -Wall -Wno-unused -Winit-self -Wpointer-arith -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Werror=date-time -DNDEBUG -g -Wstrict-aliasing=2 -DNDEBUG -UDEBUG  -D__compiler_offsetof=__builtin_offsetof -Werror=int-conversion -Wno-reserved-id-macro -Wno-format-pedantic -Wno-unused-command-line-argument -fcolor-diagnostics -nostdlibinc  -target arm-linux-androideabi    -target arm-linux-androideabi -Bprebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/arm-linux-androideabi/bin    -std=gnu99     -mthumb -Os -fomit-frame-pointer -fno-strict-aliasing   -DHAS_NO_BDROID_BUILDCFG -DSPRD_WCNBT_MARLIN -fPIC -D_USING_LIBCXX   -Werror=int-to-pointer-cast -Werror=pointer-to-int-cast  -Werror=address-of-temporary -Werror=null-dereference -Werror=return-type  -MD -MF /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d -o /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.o vendor/sprd/open-source/apps/engmode/bt/bt_cmd_executer.c ) && (cp /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.P; sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\\\\$//' -e '/^\$/ d' -e 's/\$/ :/' < /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d >> /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.P; rm -f /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d )"
In file included from vendor/sprd/open-source/apps/engmode/bt/bt_cmd_executer.c:30:
vendor/nxp-nfc/opensource/libnfc-nci/src/include/bt_types.h:22:10: fatal error: 'data_types.h' file not found
#include "data_types.h"
         ^
1 error generated.

(note that in order to speed up the development I was no longer running mka systemimage, but mka libengbt, which is the library affected by the failure). I had three candidates:

vendor/nxp-nfc/opensource/libnfc-nci/p61-jcop-kit/inc/data_types.h
vendor/nxp-nfc/opensource/libnfc-nci/src/gki/ulinux/data_types.h
vendor/nxp-nfc/opensource/libnfc-nci/halimpl/bcm2079x/gki/ulinux/data_types.h

they didn't seem to be very different, so I picked the first one. I got then another failure:

[ 35% 12/34] target thumb C: libengbt <= vendor/sprd/open-source/apps/engmode/bt/bt_cmd_executer.c
FAILED: /bin/bash -c "(PWD=/proc/self/cwd  prebuilts/clang/host/linux-x86/clang-2690385/bin/clang -I device/samsung/sharkls-common/include -I system/bt/stack/include -I vendor/nxp-nfc/opensource/libnfc-nci/src/include -I vendor/nxp-nfc/opensource/libnfc-nci/p61-jcop-kit/inc/ -I system/bt/include -I vendor/sprd/open-source/apps/engmode/bt -I /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates -I /mnt/Lavoro/halium/out/target/product/j3xlte/gen/SHARED_LIBRARIES/libengbt_intermediates -I libnativehelper/include/nativehelper \$(cat /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/import_includes) -isystem system/core/include -isystem system/media/audio/include -isystem hardware/libhardware/include -isystem hardware/libhardware_legacy/include -isystem libnativehelper/include -isystem frameworks/native/include -isystem frameworks/native/opengl/include -isystem frameworks/av/include -isystem frameworks/base/include -isystem hardware/ril/include -isystem /mnt/Lavoro/halium/out/target/product/j3xlte/obj/include -isystem bionic/libc/arch-arm/include -isystem bionic/libc/include -isystem bionic/libc/kernel/uapi -isystem bionic/libc/kernel/common -isystem bionic/libc/kernel/uapi/asm-arm -isystem bionic/libm/include -isystem bionic/libm/include/arm -c    -fno-exceptions -Wno-multichar -msoft-float -ffunction-sections -fdata-sections -funwind-tables -fstack-protector-strong -Wa,--noexecstack -Werror=format-security -D_FORTIFY_SOURCE=2 -fno-short-enums -no-canonical-prefixes -mcpu=cortex-a7 -mfpu=neon-vfpv4 -D__ARM_FEATURE_LPAE=1 -mfloat-abi=softfp -DANDROID -fmessage-length=0 -W -Wall -Wno-unused -Winit-self -Wpointer-arith -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Werror=date-time -DNDEBUG -g -Wstrict-aliasing=2 -DNDEBUG -UDEBUG  -D__compiler_offsetof=__builtin_offsetof -Werror=int-conversion -Wno-reserved-id-macro -Wno-format-pedantic -Wno-unused-command-line-argument -fcolor-diagnostics -nostdlibinc  -target arm-linux-androideabi    -target arm-linux-androideabi -Bprebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/arm-linux-androideabi/bin    -std=gnu99     -mthumb -Os -fomit-frame-pointer -fno-strict-aliasing   -DHAS_NO_BDROID_BUILDCFG -DSPRD_WCNBT_MARLIN -fPIC -D_USING_LIBCXX   -Werror=int-to-pointer-cast -Werror=pointer-to-int-cast  -Werror=address-of-temporary -Werror=null-dereference -Werror=return-type  -MD -MF /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d -o /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.o vendor/sprd/open-source/apps/engmode/bt/bt_cmd_executer.c ) && (cp /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.P; sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\\\\$//' -e '/^\$/ d' -e 's/\$/ :/' < /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d >> /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.P; rm -f /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libengbt_intermediates/bt_cmd_executer.d )"
vendor/sprd/open-source/apps/engmode/bt/bt_cmd_executer.c:31:10: fatal error: 'btm_api.h' file not found
#include "btm_api.h"
         ^
1 error generated.

I search in the internet revealed that this is part of Bluedroid, which I assume is the Android bluetooth stack; so I looked in the Halium manifest directory for a repo line:

$ git grep blue
motorola_potter.xml:    <project path="external/bluetooth/bluedroid" name="android_external_bluetooth_bluedroid" remote="los" revision="cm-12.1" />

This one seemed promising, so I added it to my manifest. I ran the setup script again, and had a look at the newly downloaded external/bluetooth/bluedroid/stack/include/ directory: that not only included the btm_api.h file, but also the bt_types.h which we were needing before. So I went back and tried removing the two include lines related to the nxp-nfc projects and replaced them with this one from bluedroid. I got the same issue about the missing data_types.h file, so I added back the vendor/nxp-nfc/opensource/libnfc-nci/p61-jcop-kit/inc line. I then got an error about a missing tb_target.h file, which is found in external/bluetooth/bluedroid/include/, so I added this line as well. This let me build libengbt successfully. The systemimage target instead failed later:

[ 12% 502/3912] target thumb C++: audio.primary.sc8830 <= vendor/sprd/modules/audio/normal/audiotest/auto_audio_v2.cpp
FAILED: /bin/bash -c "(PWD=/proc/self/cwd  prebuilts/clang/host/linux-x86/clang-2690385/bin/clang++ -I device/samsung/sharkls-common/include -I external/tinyalsa/include -I external/expat/lib -I system/media/audio_utils/include -I system/media/audio_effects/include -I vendor/sprd/open-source/apps/engmode -I vendor/sprd/modules/audio/normal/vb_pga -I vendor/sprd/modules/audio/normal/record_process -I vendor/sprd/modules/audio/normal/include -I vendor/sprd/modules/audio/normal/DumpData -I vendor/sprd/modules/audio/normal/audiotest -I vendor/sprd/modules/audio/normal/record_nr -I vendor/sprd/modules/audio/normal/skd -I vendor/sprd/modules/audio/normal/resample_api -I vendor/sprd/modules/audio/normal/custom_mmi -I vendor/sprd/modules/resampler -I vendor/sprd/modules/audio/normal/libaudioril -I vendor/sprd/open-source/libs/libatci -I vendor/sprd/modules/audio/normal/vb_effect/v2 -I vendor/sprd/modules/audio/normal -I /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/audio.primary.sc8830_intermediates -I /mnt/Lavoro/halium/out/target/product/j3xlte/gen/SHARED_LIBRARIES/audio.primary.sc8830_intermediates -I libnativehelper/include/nativehelper \$(cat /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/audio.primary.sc8830_intermediates/import_includes) -isystem system/core/include -isystem system/media/audio/include -isystem hardware/libhardware/include -isystem hardware/libhardware_legacy/include -isystem libnativehelper/include -isystem frameworks/native/include -isystem frameworks/native/opengl/include -isystem frameworks/av/include -isystem frameworks/base/include -isystem hardware/ril/include -isystem /mnt/Lavoro/halium/out/target/product/j3xlte/obj/include -isystem bionic/libc/arch-arm/include -isystem bionic/libc/include -isystem bionic/libc/kernel/uapi -isystem bionic/libc/kernel/common -isystem bionic/libc/kernel/uapi/asm-arm -isystem bionic/libm/include -isystem bionic/libm/include/arm -c    -fno-exceptions -Wno-multichar -msoft-float -ffunction-sections -fdata-sections -funwind-tables -fstack-protector-strong -Wa,--noexecstack -Werror=format-security -D_FORTIFY_SOURCE=2 -fno-short-enums -no-canonical-prefixes -mcpu=cortex-a7 -mfpu=neon-vfpv4 -D__ARM_FEATURE_LPAE=1 -mfloat-abi=softfp -DANDROID -fmessage-length=0 -W -Wall -Wno-unused -Winit-self -Wpointer-arith -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Werror=date-time -DNDEBUG -g -Wstrict-aliasing=2 -DNDEBUG -UDEBUG  -D__compiler_offsetof=__builtin_offsetof -Werror=int-conversion -Wno-reserved-id-macro -Wno-format-pedantic -Wno-unused-command-line-argument -fcolor-diagnostics -nostdlibinc  -target arm-linux-androideabi    -target arm-linux-androideabi -Bprebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/arm-linux-androideabi/bin    -fvisibility-inlines-hidden -Wsign-promo  -Wno-inconsistent-missing-override -nostdlibinc  -target arm-linux-androideabi   -mthumb -Os -fomit-frame-pointer -fno-strict-aliasing  -fno-rtti -D_POSIX_SOURCE -Wno-multichar -g -DAUDIO_SPIPE_TD -D_LPA_IRAM -DVOIP_DSP_PROCESS -DAUDIO_HAL_ANDROID_N_API -DFM_VERSION_IS_GOOGLE -fPIC -D_USING_LIBCXX -std=gnu++14  -Werror=int-to-pointer-cast -Werror=pointer-to-int-cast  -Werror=address-of-temporary -Werror=null-dereference -Werror=return-type    -MD -MF /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/audio.primary.sc8830_intermediates/audiotest/auto_audio_v2.d -o /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/audio.primary.sc8830_intermediates/audiotest/auto_audio_v2.o vendor/sprd/modules/audio/normal/audiotest/auto_audio_v2.cpp ) && (cp /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/audio.primary.sc8830_intermediates/audiotest/auto_audio_v2.d /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/audio.primary.sc8830_intermediates/audiotest/auto_audio_v2.P; sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\\\\$//' -e '/^\$/ d' -e 's/\$/ :/' < /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/audio.primary.sc8830_intermediates/audiotest/auto_audio_v2.d >> /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/audio.primary.sc8830_intermediates/audiotest/auto_audio_v2.P; rm -f /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/audio.primary.sc8830_intermediates/audiotest/auto_audio_v2.d )"
vendor/sprd/modules/audio/normal/audiotest/auto_audio_v2.cpp:21:9: warning: 'LOG_TAG' macro redefined [-Wmacro-redefined]
#define LOG_TAG "SPRD_AUDIOTRACK"
        ^
system/core/include/log/log.h:65:9: note: previous definition is here
#define LOG_TAG NULL
        ^
vendor/sprd/modules/audio/normal/audiotest/auto_audio_v2.cpp:186:39: warning: unused parameter 'user' [-Wunused-parameter]
void playerCallback( int event, void* user, void *info )
                                      ^
vendor/sprd/modules/audio/normal/audiotest/auto_audio_v2.cpp:245:9: error: use of undeclared identifier 'AUDIO_DEVICE_OUT_FM_HEADSET'; did you mean 'AUDIO_DEVICE_OUT_WIRED_HEADSET'?
        AUDIO_DEVICE_OUT_FM_HEADSET,
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~
        AUDIO_DEVICE_OUT_WIRED_HEADSET
system/media/audio/include/system/audio.h:711:5: note: 'AUDIO_DEVICE_OUT_WIRED_HEADSET' declared here
    AUDIO_DEVICE_OUT_WIRED_HEADSET             = 0x4,
    ^

This is an issue reported here, The link mentioned there is for CyanogenMod 13, but I found one for LineageOS 14.1 here. So I ran

cd device/samsung/sharkls-common/patches/
./apply_sprd-diff.sh

This failed on a couple of patches, for which I reported an issue here, though it's likely not important for us (as it mainly regards Java code). The build proceeded:

[ 10% 436/4011] target thumb C++: libstagefright <= frameworks/av/media/libstagefright/CameraSource.cpp
FAILED: /bin/bash -c "(PWD=/proc/self/cwd  prebuilts/clang/host/linux-x86/clang-2690385/bin/clang++ -I device/samsung/sharkls-common/include -I ./frameworks/av/include/media/ -I ./frameworks/av/media/libavextensions -I ./frameworks/av/media/libstagefright/mpeg2ts -I ./frameworks/av/include/media/stagefright/timedtext -I ./frameworks/native/include/media/hardware -I ./frameworks/native/include/media/openmax -I ./external/flac/include -I ./external/tremolo -I ./external/libvpx/libwebm -I ./system/netd/include -I system/media/audio_utils/include -I hardware/qcom/media/sc8830/mm-core/inc -I ./external/stagefright-plugins/include -I frameworks/av/media/libstagefright -I /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libstagefright_intermediates -I /mnt/Lavoro/halium/out/target/product/j3xlte/gen/SHARED_LIBRARIES/libstagefright_intermediates -I libnativehelper/include/nativehelper \$(cat /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libstagefright_intermediates/import_includes) -isystem system/core/include -isystem system/media/audio/include -isystem hardware/libhardware/include -isystem hardware/libhardware_legacy/include -isystem libnativehelper/include -isystem frameworks/native/include -isystem frameworks/native/opengl/include -isystem frameworks/av/include -isystem frameworks/base/include -isystem hardware/ril/include -isystem /mnt/Lavoro/halium/out/target/product/j3xlte/obj/include -isystem bionic/libc/arch-arm/include -isystem bionic/libc/include -isystem bionic/libc/kernel/uapi -isystem bionic/libc/kernel/common -isystem bionic/libc/kernel/uapi/asm-arm -isystem bionic/libm/include -isystem bionic/libm/include/arm -c    -fno-exceptions -Wno-multichar -msoft-float -ffunction-sections -fdata-sections -funwind-tables -fstack-protector-strong -Wa,--noexecstack -Werror=format-security -D_FORTIFY_SOURCE=2 -fno-short-enums -no-canonical-prefixes -mcpu=cortex-a7 -mfpu=neon-vfpv4 -D__ARM_FEATURE_LPAE=1 -mfloat-abi=softfp -DANDROID -fmessage-length=0 -W -Wall -Wno-unused -Winit-self -Wpointer-arith -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Werror=date-time -DNDEBUG -g -Wstrict-aliasing=2 -DNDEBUG -UDEBUG  -D__compiler_offsetof=__builtin_offsetof -Werror=int-conversion -Wno-reserved-id-macro -Wno-format-pedantic -Wno-unused-command-line-argument -fcolor-diagnostics -nostdlibinc  -target arm-linux-androideabi    -target arm-linux-androideabi -Bprebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/arm-linux-androideabi/bin    -fvisibility-inlines-hidden -Wsign-promo  -Wno-inconsistent-missing-override -nostdlibinc  -target arm-linux-androideabi   -mthumb -Os -fomit-frame-pointer -fno-strict-aliasing  -fno-rtti -Wno-multichar -Werror -Wno-error=deprecated-declarations -Wall -DENABLE_STAGEFRIGHT_EXPERIMENTS -DUSE_SPRD_COLORFORMAT -fPIC -D_USING_LIBCXX -fsanitize=unsigned-integer-overflow,signed-integer-overflow -fsanitize-trap=all -ftrap-function=abort -std=gnu++14  -Werror=int-to-pointer-cast -Werror=pointer-to-int-cast  -Werror=address-of-temporary -Werror=null-dereference -Werror=return-type    -MD -MF /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libstagefright_intermediates/CameraSource.d -o /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libstagefright_intermediates/CameraSource.o frameworks/av/media/libstagefright/CameraSource.cpp ) && (cp /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libstagefright_intermediates/CameraSource.d /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libstagefright_intermediates/CameraSource.P; sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\\\\$//' -e '/^\$/ d' -e 's/\$/ :/' < /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libstagefright_intermediates/CameraSource.d >> /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libstagefright_intermediates/CameraSource.P; rm -f /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libstagefright_intermediates/CameraSource.d )"
frameworks/av/media/libstagefright/CameraSource.cpp:134:16: error: use of undeclared identifier 'OMX_SPRD_COLOR_FormatYVU420SemiPlanar'; did you mean 'OMX_QCOM_COLOR_FormatYVU420SemiPlanar'?
        return OMX_SPRD_COLOR_FormatYVU420SemiPlanar;
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
               OMX_QCOM_COLOR_FormatYVU420SemiPlanar
./frameworks/native/include/media/openmax/OMX_IVCommon.h:169:5: note: 'OMX_QCOM_COLOR_FormatYVU420SemiPlanar' declared here
    OMX_QCOM_COLOR_FormatYVU420SemiPlanar = 0x7FA30C00,
    ^
1 error generated.

I realized that there were more patches to be applied, and that the script stopped as soon as it stumbled into a failing patch. So, I took all the failing patches out of the way:

mv sprd-diff/frameworks_base.diff{,.skip}
mv sprd-diff/frameworks_opt.diff{,.skip}
mv sprd-diff/packages_apps.diff{,.skip}
mv sprd-diff/packages_services.diff{,.skip}
mv sprd-diff/system_bt.diff{,.skip} # This looks potentially worth a revisit: should we have a system/bt dir?

The next build failed here:

[ 81% 3506/4283] target thumb C: libbt-vendor <= vendor/sprd/wcn/bt/libbt/src/bt_vendor_sprd.c
FAILED: /bin/bash -c "(PWD=/proc/self/cwd  prebuilts/clang/host/linux-x86/clang-2690385/bin/clang -I device/samsung/sharkls-common/include -I vendor/sprd/wcn/bt/libbt/include -I system/bt/hci/include -I vendor/sprd/wcn/bt/libbt/conf/sprd/marlin/include -I vendor/sprd/wcn/bt/libbt -I /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libbt-vendor_intermediates -I /mnt/Lavoro/halium/out/target/product/j3xlte/gen/SHARED_LIBRARIES/libbt-vendor_intermediates -I libnativehelper/include/nativehelper \$(cat /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libbt-vendor_intermediates/import_includes) -isystem system/core/include -isystem system/media/audio/include -isystem hardware/libhardware/include -isystem hardware/libhardware_legacy/include -isystem libnativehelper/include -isystem frameworks/native/include -isystem frameworks/native/opengl/include -isystem frameworks/av/include -isystem frameworks/base/include -isystem hardware/ril/include -isystem /mnt/Lavoro/halium/out/target/product/j3xlte/obj/include -isystem bionic/libc/arch-arm/include -isystem bionic/libc/include -isystem bionic/libc/kernel/uapi -isystem bionic/libc/kernel/common -isystem bionic/libc/kernel/uapi/asm-arm -isystem bionic/libm/include -isystem bionic/libm/include/arm -c    -fno-exceptions -Wno-multichar -msoft-float -ffunction-sections -fdata-sections -funwind-tables -fstack-protector-strong -Wa,--noexecstack -Werror=format-security -D_FORTIFY_SOURCE=2 -fno-short-enums -no-canonical-prefixes -mcpu=cortex-a7 -mfpu=neon-vfpv4 -D__ARM_FEATURE_LPAE=1 -mfloat-abi=softfp -DANDROID -fmessage-length=0 -W -Wall -Wno-unused -Winit-self -Wpointer-arith -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Werror=date-time -DNDEBUG -g -Wstrict-aliasing=2 -DNDEBUG -UDEBUG  -D__compiler_offsetof=__builtin_offsetof -Werror=int-conversion -Wno-reserved-id-macro -Wno-format-pedantic -Wno-unused-command-line-argument -fcolor-diagnostics -nostdlibinc  -target arm-linux-androideabi    -target arm-linux-androideabi -Bprebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/arm-linux-androideabi/bin    -std=gnu99     -mthumb -Os -fomit-frame-pointer -fno-strict-aliasing   -fPIC -D_USING_LIBCXX   -Werror=int-to-pointer-cast -Werror=pointer-to-int-cast  -Werror=address-of-temporary -Werror=null-dereference -Werror=return-type  -MD -MF /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libbt-vendor_intermediates/src/bt_vendor_sprd.d -o /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libbt-vendor_intermediates/src/bt_vendor_sprd.o vendor/sprd/wcn/bt/libbt/src/bt_vendor_sprd.c ) && (cp /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libbt-vendor_intermediates/src/bt_vendor_sprd.d /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libbt-vendor_intermediates/src/bt_vendor_sprd.P; sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\\\\$//' -e '/^\$/ d' -e 's/\$/ :/' < /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libbt-vendor_intermediates/src/bt_vendor_sprd.d >> /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libbt-vendor_intermediates/src/bt_vendor_sprd.P; rm -f /mnt/Lavoro/halium/out/target/product/j3xlte/obj/SHARED_LIBRARIES/libbt-vendor_intermediates/src/bt_vendor_sprd.d )"
In file included from vendor/sprd/wcn/bt/libbt/src/bt_vendor_sprd.c:31:
vendor/sprd/wcn/bt/libbt/include/bt_vendor_sprd.h:33:10: fatal error: 'bt_vendor_lib.h' file not found
#include "bt_vendor_lib.h"
         ^
1 error generated.

I had this file in external/bluetooth/bluedroid/hci/include/bt_vendor_lib.h, and I was about to add this directory to the vendor/sprd/wcn/bt/libbt/Android.mk file when I realized that this file was already including $(BDROID_DIR)/hci/include and defined BDROID_DIR := $(TOP_DIR)system/bt. So I understood that I should have changed my manifest file to check out the bluedroid repo under system/bt (which also meant that I was wrong in excluding the sprd-diff/system_bt.diff patch before). I also had a quick look at other manifest files, and I saw that they were typically getting system/bt from the LineageOS android_system_bt repo. Therefore I went to modify my manifest file, changing the bluedroid line to

<project path="system/bt" name="android_system_bt" remote="los" revision="cm-14.1" />

and removed the three vendor/nxp-nfc repositories I added a while before. Then I manually applied the patch in device/samsung/sharkls-common/patches/sprd-diff/system_bt.diff and I revisited the vendor/sprd/open-source/apps/engmode/bt/Android.mk to remove all the include lines I added before. The next build succeeded:

[...]
[ 99% 2250/2255] build /mnt/Lavoro/halium/out/target/product/j3xlte/obj/NOTICE.html
Combining NOTICE files into HTML
Combining NOTICE files into text
[ 99% 2254/2255] Target system fs image: /mnt/Lavoro/halium/out/target/product/j3xlte/obj/PACKAGING/systemimage_intermediates/system.img
BuildImage: in_dir = /mnt/Lavoro/halium/out/target/product/j3xlte/system, out_file = /mnt/Lavoro/halium/out/target/product/j3xlte/obj/PACKAGING/systemimage_intermediates/system.img
fs type is not ext4
Running:  mkuserimg.sh -s /mnt/Lavoro/halium/out/target/product/j3xlte/system /mnt/Lavoro/halium/out/target/product/j3xlte/obj/PACKAGING/systemimage_intermediates/system.img ext4 system 2147483648 -D /mnt/Lavoro/halium/out/target/product/j3xlte/system -L system /mnt/Lavoro/halium/out/target/product/j3xlte/root/file_contexts.bin
make_ext4fs -s -T -1 -S /mnt/Lavoro/halium/out/target/product/j3xlte/root/file_contexts.bin -L system -l 2147483648 -a system /mnt/Lavoro/halium/out/target/product/j3xlte/obj/PACKAGING/systemimage_intermediates/system.img /mnt/Lavoro/halium/out/target/product/j3xlte/system /mnt/Lavoro/halium/out/target/product/j3xlte/system
Creating filesystem with parameters:
    Size: 2147483648
    Block size: 4096
    Blocks per group: 32768
    Inodes per group: 8192
    Inode size: 256
    Journal blocks: 8192
    Label: system
    Blocks: 524288
    Block groups: 16
    Reserved block group size: 127
Created filesystem with 1381/131072 inodes and 55467/524288 blocks
Running ['mkuserimg.sh', '-s', '/mnt/Lavoro/halium/out/target/product/j3xlte/system', '/mnt/Lavoro/halium/out/target/product/j3xlte/obj/PACKAGING/systemimage_intermediates/system.img', 'ext4', 'system', '2147483648', '-D', '/mnt/Lavoro/halium/out/target/product/j3xlte/system', '-L', 'system', '/mnt/Lavoro/halium/out/target/product/j3xlte/root/file_contexts.bin'] command, exit code = 0
[ 99% 2254/2255] Construct recovery from boot
failed to reconstruct target deflate chunk 1 [(null)]; treating as normal
chunk 0: type 0 start 0 len 5582858
chunk 1: type 2 start 5582858 len 8611584
chunk 2: type 0 start 9205682 len 673886
Construct patches for 3 chunks...
patch   0 is 212 bytes (of 5582858)
patch   1 is 4311951 bytes (of 3622824)
patch   2 is 207 bytes (of 673886)
chunk   0: normal   (         0,    5582858)         212
chunk   1: deflate  (   5582858,    8574717)     4311951  (null)
chunk   2: normal   (  14157575,     674057)         207
[100% 2255/2255] Install system fs image: /mnt/Lavoro/halium/out/target/product/j3xlte/system.img
/mnt/Lavoro/halium/out/target/product/j3xlte/system.img+/mnt/Lavoro/halium/out/target/product/j3xlte/obj/PACKAGING/recovery_patch_intermediates/recovery_from_boot.p maxsize=2192424960 blocksize=135168 total=197470210 reserve=22167552
make: Leaving directory '/mnt/Lavoro/halium'

#### make completed successfully (01:22 (mm:ss)) ####

Just to be on the safe side (to make sure I didn't have stale files from previous build attempts), I ran

mka clean
mka halium-boot
LANG=C mka systemimage

again, without encountering any errors. I also ran mka hybris-boot as per the Halium documentation, as it might be a good idea to test the Halium reference image first. hybris-boot.img was built successfully at the first try.

Flashing and testing the Halium reference image

Following the Halium documentation, I downloaded the Halium rootfs image:

mkdir rootfs
cd rootfs
wget 'http://bshah.in/halium/halium-rootfs-20170630-151006.tar.gz'

A small section of the documentation is dedicated on how to install the boot image for Samsung devices; it closely matches the steps I took before to install TWRP, which is comforting.

adb reboot download    # much easier than the weird key combinations!
sudo heimdall flash --BOOT out/target/product/j3xlte/hybris-boot.img

The last command ended with this output:

Uploading BOOT
100%
BOOT upload successful

Ending session...
ERROR: Failed to receive session end confirmation!
Releasing device interface...

and the device was stuck on the same screen. I then thought of using TWRP to flash this device. I rebooted in download mode and installed TWRP as explained near the beginning of this blog post, then I uploaded my boot image to the SD card:

adb push out/target/product/j3xlte/hybris-boot.img /sdcard/

In TWRP, I did a wipe first: I wiped off the "System", "Data" and "cache" partitions (you can choose which partitions to wipe under the "Advanced wipe" option). Then I went back and pressed the "Install" button, then "Install Image" and picked my hybris-boot.img file (and chose "Boot" as target, of course). Make sure you don't reboot at this stage, or you'll lose adb access and the rest of the commands will fail.

I then installed the halium-install script:

git clone https://gitlab.com/JBBgameich/halium-install.git

I then downloaded the original Samsung ROM (see below for the link) and unzipped it in the rootfs directory I created before. That produced a file named J320FXXU0AQK2_J320FOXE0AQK2_J320FXXU0AQI1_HOME.tar.md5, from which I extracted the system.img and uploaded it to the device along with the halium rootfs, as explained in the "Halium guide":

tar xvf J320FXXU0AQK2_J320FOXE0AQK2_J320FXXU0AQI1_HOME.tar.md5 system.img
../halium-install/halium-install -p halium -v halium-rootfs-20170630-151006.tar.gz system.img

The last command complained about simg2img not being in $PATH, despite the fact that it was. A small change to the script fixed this. Once that was fixed it successfully pushed the files to the device, after asking me which password I wanted to set for the root user (on the phone): I chose a familiar "0000".

After rebooting, however, the device didn't go beyond the boot screen (with a red line warning about SELinux not being enabled). It just got stuck there, and was not even detected via USB.

Therefore, it's time to end this post with a "To be continued" message, and with a bonus section on how to restore the stock image below.

Restoring the Samsung stock ROM

The stock ROM can be downloaded from sammobile.com; you can restore the original image with this commands:

unzip J320*.zip
tar xvf J320*.tar.md5
sudo heimdall flash --KERNEL boot.img --CACHE cache.img \
    --HIDDEN hidden.img --l_fixnv2 nvitem.bin --PARAM param.lfs \
    --pm_sys PM_sharkl_arm7.bin --RECOVERY recovery.img \
    --SBOOT sboot.bin --SBOOT2 sboot2.bin --BOOT spl.img \
    --l_modem SPRDCP.img --l_gdsp SPRDGDSP.img --l_ldsp SPRDLTEDSP.img \
    --l_warm SPRDWDSP.img --SYSTEM system.img

If you were so crazy as to follow my steps, now running the command above is probably the wisest thing you could do. :-)

0 Add to favourites0 Bury

July 01 2019

13:42

Qbs and code coverage reports

You know that I'm not an early adopter. That's why it was only a couple of weeks ago when I decided to give Qbs a try, by using the good old Mappero (and its spin-off, Mappero Geotagger) as a test bench. Yes, I know that the Qt company is not going to maintain Qbs anymore in the future, but the little I knew about Qbs was enough to convince me that it's a project worth supporting. So, better late than never -- and hopefully the community (me included) will do a good job in keeping Qbs thriving.

Having Mappero build with Qbs was the simplest thing ever. The only issue I met was in building the unit tests, because I'm used to set the rpath on test executables in order to make it easy to run them uninstalled, and with qmake I achieved that with this:

QMAKE_RPATHDIR = $${QMAKE_LIBDIR}

In turns out that with Qbs you can do it in almost the same way, but for some reason I couldn't figure it out and I even reported a bug to which I got some nice suggestions, before eventually settling on this:

import qbs 1.0

Test {
    name: "path-test"

    files: [
        "path-test.cpp",
        "path-test.h",
        "paths.qrc",
    ]

    Depends { name: "Mappero" }
    cpp.rpaths: cpp.libraryPaths    // <-- this does the trick!
}

It's surprisingly similar to how it's done in qmake, so it's not clear even to me why I didn't guess that immediately. Anyway, that was literally my only problem, and you can see the whole set of Qbs files I wrote by having a look at this commit.

Given how easy the migration was, I thought I should also try to add a code coverage report; that's not something I had in my qmake build either, but it's something I really want to have in all my newer projects.

Teaching Qbs to make a code coverage report

Unfortunately, my search for examples on how to have Qbs prepare a coverage report was mostly insuccessful, but thanks to some amazing help from Christian in the #qbs IRC channel, this was not hard to achieve. So, I hope to be of some help myself too, by sharing how this works.

First of all, it must be said that Qbs doesn't know anything about code coverage, at all. However, it's possible (and often easy) to extend Qbs by adding your own Product with its own set of build rules, so here's the CoverageReport item for Mappero (though, it should be general enough to be reusable in your own project):

import qbs

Product {
    name: "coverage"

    property string outputDirectory: "coverage-html"
    property stringList extractPatterns: []

    builtByDefault: false
    files: ["**"]
    type: ["coverage.html"]

    Depends { productTypes: ["autotest-result"] }

    Rule {
        multiplex: true
        explicitlyDependsOnFromDependencies: ["autotest-result"]
        outputFileTags: "coverage.html"
        requiresInputs: false
        prepare: {
            var commands = []
            var captureCmd = new Command("lcov", [
                "--directory", project.sourceDirectory,
                "--capture",
                "--output-file", "coverage.info",
                "--no-checksum",
                "--compat-libtool",
            ]);
            captureCmd.description = "Collecting coverage data";
            captureCmd.highlight = "coverage";
            captureCmd.silent = false;
            commands.push(captureCmd);

            var extractArgs = []
            for (var i = 0; i < product.extractPatterns.length; i++) {
                extractArgs.push("--extract");
                extractArgs.push("coverage.info");
                extractArgs.push(product.extractPatterns[i]);
            }
            if (product.extractPatterns.length > 0) {
                extractArgs.push("-o");
                extractArgs.push("coverage.info");
                var extractCmd = new Command("lcov", extractArgs);
                extractCmd.description = "Extracting coverage data";
                extractCmd.highlight = "coverage";
                extractCmd.silent = false;
                commands.push(extractCmd);
            }

            var filterCmd = new Command("lcov", [
                "--remove", "coverage.info", 'moc_*.cpp',
                "--remove", "coverage.info", 'qrc_*.cpp',
                "--remove", "coverage.info", '*/tests/*',
                "-o", "coverage.info",
            ]);
            filterCmd.description = "Filtering coverage data";
            filterCmd.highlight = "coverage";
            filterCmd.silent = false;
            commands.push(filterCmd);

            var genhtmlCmd = new Command("genhtml", [
                "--prefix", project.sourceDirectory,
                "--output-directory", product.outputDirectory,
                "--title", "Code coverage",
                "--legend",
                "--show-details",
                "coverage.info",
            ]);
            genhtmlCmd.description = "Generate HTML coverage report";
            genhtmlCmd.highlight = "coverage";
            genhtmlCmd.silent = false;
            commands.push(genhtmlCmd);

            return commands;
        }
    }
}

The most important thing here are the references to the autotest-result tag: this is the tag used by the AutotestRunner Qbs item, which is responsible for running the unit tests. Referencing its product's tag in the Depends item and in the explicitlyDependsOnFromDependencies properties ensures that "building" our product will cause the unit tests to run. Other needed bits are the requiresInputs: false property, which means that our rule doesn't have any required inputs, and the builtByDefault: false property, which says that our coverage report should not be generated when just typing qbs. Instead, to run the tests and get the code coverage report one will have to request it explicitly, by typing

qbs -p coverage

The prepare property of the Rule is where the commands to generate the code coverage report are defined. Here we can use the Command item to invoke external programs, and we return a list of such items, so that the commands will be executed in sequence. Note that here I'm using lcov and expecting to find the coverage data produced by gcov, so this is probably not portable outside of Linux/gcc.

Using the CoverageReport item is quite easy: you just need to declare it, and specify which paths contain the coverage data that you are interested in (otherwise, lcov will collect data from all object files that it find under the build directory, which might not be what you desire):

    CoverageReport {
        condition: project.enableCoverage
        extractPatterns: [ '*/src/*.cpp', '*/lib/*.cpp' ]
    }

There's little more than that to be done. Of course, you need to find a way to pass the --coverage option to gcc when building your products, and for this I created a small buildconfig module in qbs/modules/buildconfig/BuildConfig.qbs which I depend on in all products which I wish to build with coverage enabled:

import qbs

Module {
    cpp.cxxFlags: project.enableCoverage ? ["--coverage"] : undefined
    cpp.dynamicLibraries: project.enableCoverage ? ["gcov"] : undefined

    Depends { name: "cpp" }
}

If all this looks scary, you should probably have a look at the diff which shows how I added code coverage reporting to qbs: hopefully you'll find that it's not that complex, after all.

I hope that Qbs users will find this interesting, and possibly improving my setup. Ideally we should try to get something like this part of Qbs itself, but portability outside of Linux / gcc is going to be an issue.

1 Add to favourites0 Bury

June 13 2019

14:59

WIP: changing the backend for contacts in Ubports

More than one year has passed since the initial announcement of my plan to investigate using a different backend for contact storage. If you want to get a better understanding of the plan, that mail is still a good read -- not much has changed since them, planning wise.

The reason for this blog post is to give a small update on what has happened since then, and as a start nothing can be better than a couple of screenshots:

Adding CardDAV accounts in the Addressbook applicationAggregated contact details from multiple sources

In other words, that means that contact synchonisation works, both with the new CardDAV protocol (for which we'll have preconfigured setups for NextCloud and OwnCloud accounts) and with Google Contacts, for which we are now using a different engine. What you see in the second screenshot (although indeed it's not obvious at all) is that the new qtcontacts-sqlite backend performs automatic contact merging based on some simple heuristics, meaning that when synchonising the same contact from multiple sources you should not happen to find a multitude of semi-identical copies of the contact, but a single one having all the aggregated details.

Before you get too excited, I have to say that this code is pre-alpha quality and that it's not even available for testing yet. The next step is indeed to setup CI so that the packaggqes get automatically built and published to a public repository, at which point I'll probably issue another update here in my blog.

The boring stuff

And now some detail for those who might wonder why this feature is not ready yet, or would like to get an idea on the time-frame for its completion.

Apart from a chronical lack of time from my part, the feature complexity is due to the large number of components involved:

  • qtcontacts-sqlite: the QtContacts backend we are migrating to. This is a backend for the QtContacts API (used by our Addressbook application) which uses a SQLite database as storage for your contacts.
  • buteo-sync-plugin-carddav: the CardDAV plugin for Buteo (our synchronisation manager). This plugin is loaded by Buteo and synchronises the contacts between a CardDAV remote source and the qtcontacts-sqlite database.
  • buteo-sync-plugins-social: a Buteo plugin which can synchronise contacts from a multitude of sources, including Google, Facebook and Vk. At the moment we only care about Google, but once this feature has landed we can easily extend it to work with the other two as well.
  • address-book-app: this is our well-known Contacts application. It needs some minor changes to adapt to the qtcontacts-sqlite backend and to support the creation of new CardDAV, NextCloud and OwnCloud accounts.
  • QtPim: the contacts and calendar API developed by the Qt project. Our Contacts application is using the front-end side of this API, and the qtcontacts-sqlite component implements the backend side. There are some improvements proposed by Jolla, which we need to include in order to support grouping contacts by their initials.

The other tricky aspect is that the first three projects are maintained by Jolla as part of the Sailfish OS, and while on one side this means that we can share the development and maintenance burden with Jolla, on the other side of the coin it means that we need to apply extra care when submitting changes, in order not to step on each other's shoes. Specifically, Sailfish OS is using a much older version of QtPim than Ubports is, and the APIs between the two versions have changes in an incompatible version, so that it's nearly impossible to have a single code base working with both versions of QtPim. Luckily git supports branches, and Chris from Jolla was kind enough to create a branch for us in their upstream repository where I've proposed our changes (and they are a lot!).

However, this is not as bad as it sounds, and the fact that I have a roughly working version on my development device is a good sign that things are moving forwards.

0 Add to favourites0 Bury

May 13 2019

17:03

Ubuntu on the Lenovo D330

The Lenovo D330 2-in-1 convertible (or netbook as we used to say) is a quite interesting device. It is based on Intels current low-power core platform, Gemini Lake (GLK), and thus offers great battery-life and a fan-less design.

This similar to what you would from an ARM based tablet. However being x86 based and Windows focused we can expect to get Ubuntu Linux running – without requiring any out-of-tree drivers or custom kernels that never get updated as we are used-to from the ARM world.
This post will be about my experiences on doing so.

For this I will use the most recent Ubuntu 19.04 release as it contains fractional scaling support, which is essential for a 10″ 1920x1200px device. Also the orientation sensor (mostly) works out of the box, when compared to the 18.04 LTS release.

Getting to the desktop

After booting the live USB, you will notice that the screen stays black. This is caused by the i915 driver not correctly setting up the internal (DSI) screen (see FDO#109267).

A quick workaround for this is to rotate the device, causes the i915 driver to re-initialise the screen, which will work at this point. Alternatively you can also suspend/ resume the device.

Wrong screen orientation

Once on the desktop, you will notice that it is rotated by 90° which is caused by a missing mount matrix entry in udev shipped with 19.04.

First we manually rotate the screen to landscape mode by opening a terminal via Ctrl+Alt+T and typing

xrandr -o right

Now we can continue to add the missing accelerometer mount matrix. Create the file /etc/udev/hwdb.d/61-sensor-local.hwdb with the following content:

# IdeaPad D330
sensor:modalias:acpi:BOSC0200*:dmi:*:svnLENOVO:pn81H3:*
    ACCEL_MOUNT_MATRIX=0, 1, 0; -1, 0, 0; 0, 0, 1

To immediately apply the changes do

sudo systemd-hwdb update
sudo udevadm trigger -v -p DEVNAME=/dev/iio:device0
sudo service iio-sensor-proxy restart

Touch input rotation

Now you can rotate the device and the screen content will be correctly oriented. However, if you try to use the touchscreen, you will notice that it only behaves correctly in portrait mode.

This is caused by a bug in GNOME/ mutter introduced in the 3.32 release. It will be fixed in the 3.32.2 point-release. In the meantime you can use the updated mutter packages, where I have backported the fix.

Enabling fractional scaling

Finally we need to magnify the UI by 50% in order not to damage our eyes. Unfortunately the fractional scaling in Ubuntu 19.04 is hidden by default. To enable it enter the following in a console

gsettings set org.gnome.mutter experimental-features "['scale-monitor-framebuffer', 'x11-randr-fractional-scaling']"

This will enable fractional scaling for both Wayland and Xorg sessions. Here, you will get a noticeable performance hit when using Xorg. However you should choose it regardless due to the following on Wayland

  • legacy X11 application (like e.g. firefox) are just scaled up and thus appear blurry
  • GNOME shell often just stops accepting any input and there is no way to restart it on Wayland
  • due to some GNOME shell design flaws (#749) the mouse cursor appears sluggish as it is updated at a lower rate compared to Xorg

This probably also helps understanding why Xorg is still default on the majority of Linux distributions.

What does not work

Surprisingly little, actually. The only remaining issue is that you will lose sound after a suspend/ resume cycle due to some bug in snd-hda-intel-realtek.

What does work

  • WiFi
  • Front & back camera
  • Docking & Undocking of the keyboard
  • Touch screen & screen rotation

State of touch UI on Linux

However there is more than getting the hardware to start when using the device on a daily basis – you will also have to fight the linux apps. Currently only GTK3 apps have some understanding of touch events – most other toolkits need updating. To estimate the time until this will happen keep in mind that touch on Linux is a niche inside a niche.

Firefox for instance does not understand touch events and you will have to manually enable them as described here.

But even then you will be able to drag the window around – thanks to client side decoration (CSD) window placement is handled by Firefox itself, but it does not react to touch events #1530070 (same applies to Chrome btw).

Next all apps continue to launch in windowed mode. On device with limited screen space, you will soon notice why everything is full-screen on Android (and Windows Tablet Mode).

Finally the GNOME on-screen keyboard feels quite sluggish and lacks visual feedback. Therefore you will often end up with missing letters.

So in summary it is not a very pleasant experience right now.

Fortunately GNOME/ Ubuntu are supposedly on it for the next release.

Until then I would recommend sticking with Windows 10; fractional scaling just works there and you will really value its on-screen keyboard and tablet mode when having the physical keyboard detached.

0 Add to favourites0 Bury

April 24 2019

19:59

A critical view on the blockchain

At the beginning of this month I participated to the foss-north conference, in Gothenburg, and took the stage to give a short presentation of the blockchain technology. Given that my talk was somehow critical of the blockchain (or rather, of the projects using it without due reason) I was prepared to receive a wave of negative remarks, assuming that all the hype surrounding this technology would have infected a good part of my audience as well. I was therefore positively surprised when several people came to me afterwords to express their appreciation for my speech, appreciation that now makes me confident enough to share the video of the presentation here too:

I want to publicly thank Johan Thelin and all the other foss-north staff and volunteers who organized such a successful conference. They also managed to get the video recordings out in a surprisingly short time. Indeed, the above video is taken from the foss-north YouTube channel, which I recommend you to visit as there were a lot of good talks at the conference; the topics were so varied, that I'm sure you'll find at least a couple of talks of your interest.

1 Add to favourites0 Bury
Older posts are this way If this message doesn't go away, click anywhere on the page to continue loading posts.
Could not load more posts
Maybe Soup is currently being updated? I'll try again automatically in a few seconds...
Just a second, loading more posts...
You've reached the end.
(PRO)
No Soup for you

Don't be the product, buy the product!

close
YES, I want to SOUP ●UP for ...