Posts

Embedded Linux won't reboot - how to fix and repair

I have a lot of embedded systems running in our lab or in my home, all of them either as Raspberries or selfmade PCB with Yocto. Sometimes I can't reboot some systems, I think its the journald which causes some issues with SSD Cards, the error-message usually is:
Failed to open /dev/initctl

Anyhow, if you have this issue - a reboot can be force-forced:
systemctl --force --force reboot

Since a forced reboot does not sync the journal, the system typically comes up with a damaged FS. The remote fsck can be initiated by extending the command above with

sudo tune2fs -i 1m /dev/DISK && touch /forcefsck && systemctl --force --force reboot

(assumed you have access to a shell, via SSH or local access). When all goes fine, the system comes up with a clean FS. All this fuss comes from the SSD r/w actions, a well designed IoT embedded devices should have a flash mem part for the running OS.

How to fix "Problem with MergeList /var/lib/apt/lists/"

Sometimes, and more often, an update on RaspberryPi hangs with: "Problem with MergeList ..."
sudo mv /var/lib/apt/extended_states /var/lib/apt/extended_states_tmp && rm -rf /var/lib/apt/lists/* -vf && sudo apt-get update

solves the issue.


Kubernetes - delete all pods with a certain status

"How to delete all pods with a status like Error" - that is a question I often get. The DevOps answer is a one-liner, filtering by status.reason and the status we want to query, es example:

kubectl get po -A --all-namespaces -o json | jq '.items[] | select(.status.reason!=null) | select(.status.reason | contains("Evicted")) | "kubectl delete po \(.metadata.name) -n \(.metadata.namespace)"' | xargs -n 1 bash -c
In that case we delete all pods with the Status "Evicted". But it works for every status based filtering.

Why I ditched my MacBook and switched to Windows 10

I have to admit, I'm an Apple guy. I had everything, from Pixel books over Pixel phones, Blackberry, One Plus, Google Home, Amazon Fire, Alexa, Surface - but at the end I always ended up in Apple's universe. Apple TV, Home Pod, iPhone, Mac Book, Watch, Earphones.
Hey, not Mac Book anymore. That was my most annoying experience with Apple - a 2 year old MBP 15inch max configuration, worth 3.8k EUR, got the flicker bug. I wrote about in a blog post before. Apple did not recognize that as guarantee, so I sold that over eBay and went to Dell XPS. Long story short.

Now, as a windows hater and open source follower I tinkered 2 weeks or so to get Linux running, ended up with Ubuntu and POP!OS additions. Nice working horse, but - the battery life was not a life, it was a short draining nightmare. Even in standby, lid closed, the battery was empty after one night. That can Apple much better, I have to admit. So I googled, read Reddit and HackerNews and decided yesterday evening to ditch…

Dell XP 13 7390 late 2019 and Linux

Image
Since my 3 years old Macbook Pro 15inch got the flickering bug I decided to buy a Dell laptop. Why? I have a really old Dell laptop from 2004! which works perfectly as a OpenBSD firewall and internet router in my weekend house.

After I got my machine, first thing was to install PopOS. Or better, I tried. It never worked, I think it's a bug in the loader section of PopOS, a bugreport is open. After some hours of hacking and fixing I got a bit bored and used a Ubuntu installation, which worked out of the box perfectly. Yes I know I had could bought the XPS at Dell with Ubuntu on it. But I did not want to ;)

Here now some tricks to get the XPS smooth to work with - right now I have the book attached to my curved 4k monitor and code some infrastructure stuff for infinimesh.

1. Bluetooth mouse
First time a BT mouse is connected it lags. Moving the mouse, the pointer follows after seconds, slow and unresponsive. Fix by sudo vi /var/lib/bluetooth/<MAC Adapter>/<MAC mouse>:

[Co…

Kubernetes - on which node runs what?

Just had to find the node for a dedicated pod, found this line useful:

kubectl get pod -o=custom-columns=NAME:.metadata.name,STATUS:.status.phase,NODE:.spec.nodeName --all-namespaces


OSX - iCloud accounts broken after update to Catalina

Yay - that was happen to me. Magically I had two iCloud accounts, both not working - and no way to delete one of them via Settings -> Internet Accounts. After a few hours of googling, debugging I got it working again. And as usual I'd like to share that solution, I think I'm not the only one with that problem ;)
Be aware, that's a bomb solution, means it will reset all internet accounts - but also unfreeze all the broken stuff:

sudo -v ; killall -9 accountsd com.apple.iCloudHelper ; defaults delete MobileMeAccounts ; mkdir ~/Library/Accounts/Backup ; mv ~/Library/Accounts/*.sqlite* ~/Library/Accounts/Backup/ ; killall -9 accountsd com.apple.iCloudHelper ; sudo reboot

So, why does that happen? Apple uses sqlite as backend for storing the informations, and sometimes this single instance get corrupted. Have fun!