reboot: Segmentation Fault

I have no idea how I ended up in that situation, but the entire system of my odroid locked up earlier this week. Not even rebooting was working. Most of the commands failed with a segfault or bus errors.

root@odroid:~# reboot -f -r now
Segmentation fault

After a quick search, I stumbled over the Unix&Linux Stackexchange where someone says – Try to reboot with the magic SysRq key. Wait what?

echo b > /proc/sysrq-trigger

The SysRq-trigger has a ton of use cases that are documented in the kernel documentation

the b flag triggers an immediate reboot of the system without syncing or unmounting your disks. And it worked like a charm. 🎉

Angelesen #73

I listened to a podcast with Seth Godin lately and found a very matching quote:

There’s no such thing as writer’s block – there’s just bad practice.

Seth Godin on Imposter Syndrome, Getting Unstuck and The Practice

And I feel I should get back into the habit of reading… like offline dead-trees-with letters on it reading. Worked so well in 2016 (shoot time flies!) so why not this year? Anyways back to the links of the week which may be a bit Kubernetes (k8s) heavy mainly due to my self-inflicted deep dive into the world of ARM64 computing (which is great and painful, but mostly great).

FRONTEX FILES (frontexfiles.eu)

»Frontex does not meet with lobbyists.« Since Frontex is not living up to its responsibilities as an EU agency, ZDF Magazin Royale is lending a hand. We hereby present the FRONTEX FILES, the first lobbying transparency register of Frontex, the European Border and Coast Guard Agency.

Great leak done via the FOIA / IFG! On this note: Abolish Frontex!

Starship | SN9 | High-Altitude Flight Test (youtube.com)

Reminder. This is a test flight. It’s the 2nd time starship flew in this configuration. We got a lot of data and the primary objective to demonstrate control of the vehicle and subsonic recently looked very good.

/me writes this down for the next post-mortem.

upptime/upptime: ⬆️ Uptime monitor and status page powered by GitHub Actions, Issues, and Pages (github.com)

Upptime (https://upptime.js.org) is the open-source uptime monitor and status page, powered entirely by GitHub Actions and Issues. It’s made with 💚 by your friends at Koj.

Tried this out a few weeks and quite like the uptime stats. Obviously, it’s nowhere near specialized monitoring solutions but it’s a really good start for simpler projects.

ineedtope/ineedtope (github.com)

I’m always fighting with OSM to find the right filters. Maybe I’ll pull the code down and render the map for Switzerland.

↬ Habi via stumbling around on the GitHub Startpage and looking at things my friends give stars to

Dir dumme Souhüng (nidi3.github.io)

Swearing in Swissgerman at it’s best!

kubermatic/kubermatic: Kubermatic Kubernetes Platform – the Central Kubernetes Management Platform For Any Infrastructure (github.com)

Kubermatic Kubernetes Platform is in an open source project to centrally manage the global automation of thousands of Kubernetes clusters across multicloud, on-prem and edge with unparalleled density and resilience.

This looks super promising – Will put that on my list of tools to look into when playing around with k8s

k8dash (k8dash.io)

A pretty promising k8s Dashboard.

Tuning application on GKE — Drupal with MySQL (medium.com)

Switch CloudSQL -> MySQL HA on GKE One of the biggest benefits of going into the cloud is having a database as a service. With Google offering MySQL with master-slave replication and quick failover, this is the first setup most of the companies will start with. Drupal applications are very ‘database-heavy’ and do a lot of reads and writes in single user page load. Even after removing SQLProxy sidecar and using a private database connection, we still had huge issues with latency. This was caused by the database as a service being located not on the same virtual machines as Kubernetes, so even a few ms latency multiplied by many calls is making a huge difference. We have decided to try a multi-master Galera setup on GKE. We also evaluated Vitess.io (database used by YouTube service), but we have decided to stay with the solution we know.

There’s much to discover when moving an application to the Cloud – Adding Galera in the Cluster sounds like a good idea at first but gets very complicated to handle at scale. This is why DIY is hard. But moving Applications to modern (read k8s) and distributed infrastructure always comes at a price of complexity and performance trade-offs.

aquasecurity/kube-hunter: Hunt for security weaknesses in Kubernetes clusters (github.com)

kube-hunter hunts for security weaknesses in Kubernetes clusters. The tool was developed to increase awareness and visibility for security issues in Kubernetes environments.

Great tool for a quick spot check on your k8s clusters.

Otomi is like OSX for Kubernetes (otomi.io)

Otomi Container Platform offers an out-of-the-box enterprise container management platform (on top of Kubernetes) that increases developer efficiency and reduces complexity. It is a turnkey cloud native solution that integrates upstream Kubernetes with proven open source components. It is a single deployable package with curated industry proven applications and policies for better governance and security.

Otomi looks also interesting – This is also on my list to play around at some point 🙂

The latest Japanese full-frame cameras market share report from BCN Ranking (nikonrumors.com)

It’s insane to see how a big player like Nikon gets absolutely trashed by Sony.

Email from Jeff Bezos to employees (aboutamazon.com)

I intend to focus my energies and attention on new products and early initiatives.

That’s one way to make a commitment to OpenDistro 😉

Useful Tools for Better Kubernetes Development (yitaek.medium.com)

Such a long list of tools! Lens is a no-brainer for me. I use it almost daily to jump between a number of k8s clusters.

Most likely on my "things to look into list":

  • Goldilocks
  • Polaris
  • kube-ps1
  • kubectx + kubens (if you are not using those install them right now)
  • Oh my and Kubecost makes it on the top of my list due to obvious reasons!

↬ Vincenzo

SNB verdient mit GameStop-Aktien kurz mal 43 Mio. (insideparadeplatz.ch)

💎👐 all the way to the 🚀🌔

WhatsApp and the domestication of users (seirdy.one)

WhatsApp rose by trapping previously-free beings in their corral and changing their habits to create dependence on masters. Over time, this made it difficult or impossible to return to their previous lifestyle. That process should sound familiar: it’s eerily similar to the domestication of animals. I call this type of vendor lock-in user domestication: the removal of user autonomy to trap users into serving vendors.

Longread of the week on WhatsApp – Well Use Signal/Threema. Use Tor.

Angelesen #72

Didn’t find time over the weekend for the link dump so here we go! Special Monday edition so to speak.

Authorities plan to mass-uninstall Emotet from infected hosts on April 25, 2021 | ZDNet (zdnet.com)

According to public reports, also confirmed by ZDNet with two cyber-security firms that have historically tracked Emotet operations, this update contains a time-bomb-like code that will uninstall the Emotet malware on April 25, 2021, at 12:00, the local time of each computer.

Wow globally killing Emotet by taking over C&C Servers!

South African government releases its own browser just to re-enable Flash support (zdnet.com)

The South African Revenue Service has released this week its own custom web browser for the sole purpose of re-enabling Adobe Flash Player support, rather than port its existing website from using Flash to HTML-based web forms.

Flash is dead – Longe live Flash! Claps slowly, flips a small table and minds his own business now.

The lamps you’re not allowed to have. Exploring the Dubai lamps (youtube.com)

Backblaze Hard Drive Stats for 2020 (backblaze.com)

Toshiba Delivers More Zeros The new Toshiba 14TB drive (model: MG07ACA14TA) and the new Toshiba 16TB (model: MG08ACA16TEY) were introduced to our data centers in 2020 and they are putting up zeros, as in zero failures. While each drive model has only been installed for about two months, they are off to a great start.

Looks like my next drives should be a few Toshibas

↬ Sean

Enterprise Open Source – Cracks in the Foundations (youtube.com)

If you didn’t follow last weeks, Elastic-Licence-Meltdown watch those 8 minutes and watch Jeff explain it in simple terms.

apozas/contactdiary: Keep track of the people you have recently been with (github.com)

Contact Diary is a simple app that allows you to keep a record of the people you have met and the events you have attended to recently. In the undesired event that you test positive for COVID-19, Contact Diary is a place where to quickly find who you must warn about the situation.

If you try to keep track of the people you meet and are on android – this is the app you are looking for.

Österreich: Der Druck auf das Bundesamt für Verfassungsschutz nimmt zu (sueddeutsche.de)

Einer der beiden Fluchthelfer von Marsalek war offenbar der Ex-FPÖ-Nationalratsabgeordnete Thomas Schellenbacher, der seit Jahren wegen Korruptionsvorwürfen im Visier der Ermittler steht und bereits eingeräumt hat, für Marsalek einen Flug nach Minsk organisiert zu haben.

Ach? Alles Einzelfälle!

3 Common Virtual Event Problems (and How to Fix Them) (theeventscalendar.com)

Send an email an hour before your event starts, and another email right when the event begins. Always remember to include the link to the call or livestream. Consider taking advantage of other channels for reminders, like using push notifications in a browser or text reminders.

Oh yes please – Make it as easy as possible for people to join your conference!

↬ mona

How to solve conflict in a community of equals (media.ccc.de)

Good conflict resolution talk for hackerspaces and other communities of equals

Inside Huawei (republik.ch)

Joe und Sam waren bei Huawei Schweiz angestellt. Doch dann verliessen sie die chinesische Techfirma – weil sie genug hatten von der militärischen Kultur. Ein seltener Einblick in eine Arbeitswelt, in der geopolitische Ziele über allem stehen.

Longread der Woche – Dauert gut so 30 Minuten

Angelesen #71

TL;DR This might get a bit ElastiSnafuHeavy. Also, Use Signal/Threema. Use Tor 😉

εxodus – The privacy audit platform for Android applications (reports.exodus-privacy.eu.org)

εxodus analyzes Android applications in order to list the embedded trackers. A tracker is a piece of software meant to collect data about you or your usages. So, εxodus reports tell you what are the ingredients of the cake.

I traced down a specific telemetry tracker that performed around 3000 DNS Lookups for a domain per day. After playing back and forth, I figured out that it comes from 1 app with a tracker embedded. Thanks to the Exodus project, I was able to find the tracker and switch off background data for the app – Look mom no DNS Requests anymore!

"I am real mad about the Elastic relicense so I’m going to vent a bit." (twitter.com)

By using an open source license and accepting contributions, they asked the community to trust them with their CLA. Implicitly: not just at the time the public made contributions, but for the life of the project! Instead they chose to set that trust, and their community, on fire.

Elasticsearch Licencing Change #1

I remember signing the Contributor License Agreement – And also remember that it basically made me lose interest in contributing to the project. That’s why I cherish the existence of Institutions like the CNCF as a solid steward for all things Cloud Native.

PostgreSQL on ARM-based AWS EC2 Instances: Is It Any Good? (percona.com)

There were not many cases where the ARM instance becomes slower than the x86 instance in the tests we performed. The test results were consistent throughout the testing of the last couple of days. While ARM-based instance is 25 percent cheaper, it is able to show a 15-20% performance gain in most of the tests over the corresponding x86 based instances. So ARM-based instances are giving conclusively better price-performance in all aspects. We should expect more and more cloud providers to provide ARM-based instances in the future. Please let us know if you wish to see any different type of benchmark tests.

ARM-based CPUs will most likely take over the Datacenter market quite quickly. At amazee.io we started the move away from Intel to AMD CPUs in most of our infrastructure because of the Performance/Price benefit, and it won’t be much different as ARM-based Instances start to be more widely available in the cloud.

What You Should Know Before Leaking a Zoom Meeting (theintercept.com)

It is not immediately apparent at what point Zoom injects its “ultrasonic” audio watermark into the audio stream — whether this happens only if a meeting attendee presses the Record button in Zoom or if the audio stream is watermarked prior to that point. Nonetheless, when recording a Zoom meeting, it is best to avoid using Zoom’s built-in recording option and to capture the meeting using a third-party audio/video recorder. Zoom mentions that in order to identify the participant who recorded the meeting, they need at least two minutes of audio from the meeting, though it stands to reason that shorter snippets may also be identifiable if they happen to contain the audio watermark.

A few good pointers when it comes to Zooms recording fingerprinting and leaking information.

Moxie Marlinspike Has a Plan to Reclaim Our Privacy (newyorker.com)

That’s the Longread of the week folks! (You can also listen to it there’s a full recording of the article).

And as always: Us Signal/Threema. Use Tor.

Truly Doubling Down on Open Source (logz.io)

Over the last few days, we’ve been closely collaborating with a growing number of organizations that believe that Elasticsearch and Kibana need to stay open-source, Apache 2, to serve the broad and diverse community of users working with and contributing to it

Elasticsearch Licencing Change #1

Stepping up for a truly open source Elasticsearch (aws.amazon.com)

The term “open source” has had a specific meaning since it was coined in 1998. Elastic’s assertions that the SSPL is “free and open” are misleading and wrong. They’re trying to claim the benefits of open source, while chipping away at the very definition of open source itself. Their choice of SSPL belies this. SSPL is a non-open source license designed to look like an open source license, blurring the lines between the two. As the Fedora community states, “[to] consider the SSPL to be ‘Free’ or ‘Open Source’ causes [a] shadow to be cast across all other licenses in the FOSS ecosystem.”

Elasticsearch Licencing Change #3

Tobias has a great sum-up of all the important articles in his weekly newsletter.

What Parler Saw During the Attack on the Capitol (projects.propublica.org)

Below is a collection of more than 500 videos that ProPublica determined were taken during the events of Jan. 6 and were relevant and newsworthy. Taken together, they provide one of the most comprehensive records of a dark event in American history through the eyes of those who took part.

The Revolution Will Not Be Televised Livestreamed. Also wearing a mask woudln’t be stupid. You know err… the rona.

System Separation in the Continental Europe Synchronous Area on 8 January 2021 – update (entsoe.eu)

At approximately 14:05 CET, the frequency in the North-West Area of Continental Europe initially decreased to a value of 49.74 Hz within a period of around 15 seconds. Afterwards, the frequency reached a steady state value of approximately 49.84 Hz. At the same time, the frequency in the South-East Area initially increased to a value of up to 50.6 Hz before settling at a steady state frequency between 50.2 Hz and 50.3 Hz. This can also be seen from the graph below.

Due to the underfrequency in the North-West Area, the contracted interruptible services in France and Italy, in total around 1.7 GW, were disconnected in order to reduce the frequency deviation. These services are large customers, who are contracted by the respective Transmission System Operators (TSOs) and get disconnected if frequency drops under a certain threshold. In addition, 420 MW and 60 MW of supportive power were automatically activated from the Nordic and Great Britain Synchronous areas respectively. These countermeasures ensured that, already at 14:09 CET, the frequency deviation was limited to a deviation of around 0.1 Hz in the North-West Area from the nominal frequency of 50 Hz (see Figure 2).

More info on what’s known about the big Electricity grid issue that happened on 2021-01-08.

Also a great thread (in german) on why renewables aren’t to blame in this case.

WhatsApp delays privacy changes following backlash (dw.com)

Following WhatsApp’s initial announcement many users have signed up for other messaging services, including privacy-minded Signal and Telegram. Signal said that a massive influx of users had led to technical difficulties in delivering some messages on Friday.

WhatsApp also canceled its February 8 deadline for accepting the tweak to its terms of service, involving sharing data with Facebook servers.

Well totally didn’t see that one coming! I would love to see how many people moved platforms and the impact on WhatsApp. Signal/Threema/Telegram all saw a huge spike of signups.

Again Use Signal/Threema 😉

New Gear: Odroid N2+

At one of the CommunityRack.org locations, I housed an Intel NUC for several years. As the Intel Celeron asks for retirement and ARM is all the rage, I started to look at new devices that could replace the NUC.

This project’s main goals were to lower power consumption, be a single-board computer, expandable storage options, and run a stock Ubuntu.

I’ve looked at the Rock Pi N10Odrdoid N2+ and the Rock Pi 4. All good and solid boards but Ben had many good things to tell about the Hardkernel Odroid Boards when we talked about Single Board Computers. Mainly because they can pack a great amount CPU cores onto a small form factor computer.

The Rock Pi4 would have had an M.2 slot to add an SSD, but Ubuntu would only run on the C-Type of the board which was a bit complicated to get at the time of my evaluation. So I went with the Odroid and opted to use a 128G SD-Card with the possibility to move towards an EMMC Storage module later on.

The Odroid setup for now:

  • Board: Odroid N2+ & Plastic Case
  • System “Disk”: SanDisk 128GB A2 – 90MB/s write, 160 MB/s read
  • Storage: Samsung T5 SSD – 1TB
  • Power Supply: Generic 12V/2A Power supply

Getting started was easy and very RaspberryPi like. Flashed the Ubuntu Image found in the Odroid wiki and it booted without any issues.

I ran into the issue that in the Ubuntu Image available back in November some of the Kernel modules weren’t shipped. Therefore, the Docker daemon just failed to start.

I got around this by just symlinking kernels modules into the new place – Docker didn’t start with a few errors on missing br_netfilter and bridge modules. The easiest way to figure out why Docker didn’t start was running /usr/bin/dockerd directly and observing the errors. I think this has been fixed already but if someone else runs into this. That’s one very hacky way to solve the issue.

cd /lib/modules/
ln -s 4.9.236-104 4.9.230-89
modprobe bridge
modprobe br_netfilter

I’m super happy so far with the power usage, which is about 6-7 Watts fully loaded. Having 6 CPU cores if I would need those are also quite impressive for such a small form factor.

I’ll follow up with another article on the challenges I saw going ARM because there are a few gotchas along the way.