Skip to content →

Amir Eslampanah Posts

Complete Conversion from ext4 to Btrfs on Arch Linux (and other Linux Distros)

Btrfs is a file-system that touts many improvements over traditionally simple file systems like ext4. Such as but not limited to: Snapshots, and file error corruption detection and recovery

Thing is if you’ve already got an installation it’d be a pain to reinstall everything just to switch file systems; so let’s convert it 🙂

First we need to boot from a live-system eg: CD or USB, we’ll be using https://endeavouros.com/ here but any distro that comes with btrfs-convert utility should work.

Next you’ll need to find the partition that you’ve installed the OS to,

sudo su
fdisk -l

In my case its on /dev/sda2

The next step is to check the filesystem for corruption, this can be done using the fsck utility.

fsck /dev/sda2

If it’s clean then you’re good to continue, otherwise repair the partition first

Now lets do the actual conversion, this may take a long time and possibly crash if the volume is very big (but is reversible):

btrfs-convert /dev/sda2

In my case it completed successfully, so next lets mount and chroot into our OS

mount /dev/sda2 /mnt
mount -t proc none /mnt/proc
mount -t sysfs none /mnt/sys
mount -o bind /dev /mnt/dev
mount -o bind /dev/pts /mnt/dev/pts 
modprobe efivarfs
chroot /mnt bash
mount -t efivarfs efivarfs /sys/firmware/efi/efivars

Now we need to see what the filesystem uuid is for our new filesystem

blkid

In our case we can see that our /dev/sda2 has a UUID of 0dafc705-81ac-4403-99cd-8d665bb4dd3e so that’s what we will use

Next lets modify our fstab to match this (remember to change the file-system from ext4 to btrfs in the next column and set subvolid=5)

nano /etc/fstab

Next we will need to rebuild grub, first mount the EFI folder (in our case its /dev/sda1)

mount -t auto /dev/sda1 /efi

Next lets detect the operating systems on the root file-system

os-prober

Copy them to grub

grub-mkconfig > /boot/grub/grub.cfg

Now lets install grub

grub-install --efi-directory=/efi --target=x86_64-efi /dev/sda

Last but not least we may need to rebuild the initramfs (note that you may have to reboot into fallback initramfs if this command doesn’t work in chroot and run it there)

mkinitcpio -P
exit
reboot

If all went well we should have a working Btrfs installation now 🙂

and we didn’t need to re-install our programs 😛

Assuming things went well lets delete the conversion image since we don’t need it anymore (optional – saves space).

btrfs subvolume delete /ext2_saved

Manual Method: How to easily create and mount a snapshot

btrfs subvolume snapshot / @

Next we need to find out the subvolume id of this new subvolume

sudo btrfs subvolume show /@ | grep "Subvolume ID"
btrfs subvol set-default 425 /

Reboot, and in order to go back we can just mount the previous volume

btrfs subvol set-default 5 /

Automated Method:

If you don’t feel like managing the snapshots yourself, you can always use snapshot manager like snapper:

yay -S snapper snap-sync snap-pac snapper-gui-git
sudo snapper create-config /
sudo snapper list
sudo snapper-gui

See also:

http://snapper.io/tutorial.html and https://github.com/ricardomv/snapper-gui

Leave a Comment

Linux Network Optimization

sudo nano /etc/sysctl.d/boost.conf

Put

net.ipv4.tcp_rfc1337=1
net.ipv4.tcp_window_scaling=1
net.ipv4.tcp_workaround_signed_windows=1
net.ipv4.tcp_sack=1
net.ipv4.tcp_fack=1
net.ipv4.tcp_low_latency=1
net.ipv4.ip_no_pmtu_disc=0
net.ipv4.tcp_mtu_probing=1
net.ipv4.tcp_frto=2
net.ipv4.tcp_frto_response=2
net.ipv4.tcp_congestion_control=illinois
net.core.netdev_budget=600
net.core.wmem_default=262144
net.core.wmem_max=4194304
net.core.rmem_default=262144
net.core.rmem_max=4194304
net.ipv4.tcp_rmem=65535 131072 4194304
net.ipv4.tcp_wmem=65535 131072 194304
net.core.somaxconn=20489
net.ipv4.tcp_moderate_rcvbuf=1

Save and load using

sudo sysctl --system

See also: tuned package

Leave a Comment

Creating a Bridged Virtual Machine Network with Gnome-Boxes and Virt-Manager on Arch Linux

Sometimes its useful to have the host of a virtual machine to be able to communicate directly with the virtual machine’s network.

Step 0 : Create the QEMU Bridge Configuration

Next lets create the qemu bridged configuration:

sudo pacman -S qemu
sudo nano /etc/qemu/bridge.conf
sudo chmod 755 -R /etc/qemu

Put the following inside

allow virbr0
sudo chmod u+s /usr/lib/qemu/qemu-bridge-helper

Next lets allow normal users to execute the qemu-bridge-helper

Step 1 : Install the prerequisites

Lets install the required packages

sudo pacman -S gnome-boxes virt-manager dnsmasq iptables iptables-nft

Note that if you cannot find one of the above packages its a good idea to update your mirrors (see: https://wiki.archlinux.org/title/mirrors)

Step 2 : Setup Lib-Virt Daemon

Lets enable the lib-virt daemon to auto-start

systemctl enable --now libvirtd

Next lets setup the kvm group so that it can manage the libvirtd

nano /etc/polkit-1/rules.d/50-libvirt.rules

Put the following inside

/* Allow users in kvm group to manage the libvirt
daemon without authentication */
polkit.addRule(function(action, subject) {
    if (action.id == "org.libvirt.unix.manage" &&
        subject.isInGroup("kvm")) {
            return polkit.Result.YES;
    }
});

Next lets create the libvirt group if it doesn’t exist already and add ourselves to it

groupadd libvirt
sudo gpasswd -a $(whoami) libvirt

Step 3 : Configure Networking

Next lets check the virsh interface status

sudo virsh net-list --all

If default exists then start it and set it up to autostart using:

sudo virsh net-start default
sudo virsh net-autostart default

If default doesn’t exist then do this first and then the previous:

sudo virsh net-define /usr/share/libvirt/networks/default.xml

Next start up virt-manager and connect to KVM/QEMU

Run ifconfig to see if things went well. If they do then you should have a virbr0 interface now:

Now if you boot up the VM on gnome-boxes you should hopefully see an IP address on the same SUBNET.

Leave a Comment

Creating a Website Scraper using Gecko Driver (for Firefox) and Selenium (on Pop Os Linux)

First install python and pip:

sudo apt-get update
sudo apt-get install python3 python3-pip

Now lets setup a virtual environment

pip3 install virtualenv

You may get a warning about the path, add the path as follows:

sudo nano /etc/environment
sudo source /etc/environment

Now lets create the virtual environment and install selenium

mkdir -pv selenium-firefox/drivers
virtualenv .venv
source .venv/bin/activate
pip3 install selenium pandas

Now download and extract the latest gecko driver from https://github.com/mozilla/geckodriver/releases/

wget https://github.com/mozilla/geckodriver/releases/download/v0.29.1/geckodriver-v0.29.1-linux64.tar.gz
tar -xzf geckodriver-v0.29.1-linux64.tar.gz -C drivers/

Now lets create a sample script (a simple download-er):

from selenium import webdriver
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.common.keys import Keys
import time

firefoxOptions = Options()

#firefoxOptions.add_argument("-headless")
driver = webdriver.Firefox(executable_path="./drivers/geckodriver", options=firefoxOptions)

#Navigate to the login page
driver.get("https://some-page/my-account/")

time.sleep(5)

#Login
username = driver.find_element_by_id("username")
username.clear()
username.send_keys("usernamehere")

password = driver.find_element_by_id("password")
password.clear()
password.send_keys("passwordhere")

persistLogin = driver.find_element_by_id("rememberme")
persistLogin.click()

time.sleep(5)

driver.find_element_by_name("login").click()

time.sleep(5)

#Head to assets page
driver.get("downloadurlhere")

condition = True
while condition:
	# loop body here
	try: 
		downloadList = driver.find_elements_by_id('download-single-form')
		
		for download in downloadList:
			download.submit()
			time.sleep(25)

		driver.find_element_by_css_selector(".next[value='next']")
	except:
		break
# end of loop

driver.quit()

Save as browser.py and run as follows:

python3 browser.py

Note that you can use pandas to do data manipulation if necessary.

2 Comments

Configuration of Automatic Updates on Amazon Linux Version 1 & 2

The Amazon Linux AMI is a supported and maintained Linux image provided by Amazon Web Services for use on Amazon Elastic Compute Cloud (Amazon EC2). It is designed to provide a stable, secure, and high performance execution environment for applications running on Amazon EC2.

However, for whatever reason it is not enabled with automatic security updates on by default.

So lets fix that.

Step 0:

First lets install a version locking system for yum package manager. The reason we want something like this is to lock any and all packages that are too sensitive for automatic updates. A good example of this is any package where the version number can affect functionality in a major way such as gcc or Java.

sudo yum install yum-plugin-versionlock

You can then lock any package/set of packages with a statement like the following

sudo yum versionlock java*

To view a list of the current package locks, you can

sudo yum versionlock list

To remove all current package locks

sudo yum versionlock clear

Step 1:

Next lets configure automatic updates by first installing yum-cron

sudo yum install yum-cron

Lets modify the configuration to only enable security updates

sudo nano /etc/yum/yum-cron.conf

In the commands section set update_cmd to security and save-exit (Ctrl+o + Ctrl+x)

This will do something akin to

sudo yum update --security

Now lets enable it to start

sudo service yum-cron start

On AMI Version 1 also run:

sudo chkconfig yum-cron on

On AMI Version 2 also run:

sudo systemctl enable yum-cron

And you’re done!

Leave a Comment

Mounting S3 Buckets on Ubuntu/Pop Os Linux

First lets install the open source s3 Fuse

sudo apt-get install s3fs

Next you’ll need an access key id and secret access key; and you can get one from here

https://console.aws.amazon.com/iam/home?#/security_credentials

We’ll create a password file using these credentials

echo ACCESS_KEY:SECRET_ACCESS_KEY > ~/aws_s3.key
chmod 600 ~/aws_s3.key

The next step is to create the mounting point directory

mkdir ~/mount/s3-bucket

Now for the actual mount command

s3fs bucketname ~/mount/s3-bucket -o passwd_file=~/.aws_s3.key

Now you should be able to see the mounted files.

Leave a Comment

Installing Microsoft Teams on Ubuntu/Pop Os Linux

First lets add the repository key for the official release of Microsoft Teams for Linux

curl https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -

And lets add the repository

sudo sh -c 'echo "deb [arch=amd64] https://packages.microsoft.com/repos/ms-teams stable main" > /etc/apt/sources.list.d/teams.list'

Update and install teams

sudo apt install teams

Now you can login and use Microsoft teams!

However, you may encounter audio issues such as but not limited to:

  • No audio heard from call
  • Microphone cannot be heard in a call
  • Incorrect audio device is being used

These things can happen because for what ever reason Microsoft Teams for Linux is very bad at picking audio devices.

For this reason, I recommend installing PulseAudio Volume Control

sudo apt-get install pavucontrol

Then you can use the PulseAudio Volume Control to disable outputs and inputs you are not currently using, that way, teams can’t choose the wrong one.

Leave a Comment

IntelliJ Data Views & Java Collections

By default in IntelliJ IDE the internal view of a collection is hidden.

This is rather useless when debugging complex custom collections; so to avoid a bunch of extra debug code, you can simply disable the alternate view of collections in the customize data views option. (Ctrl+Shift+A)

You can also enable showing of static fields in here too which I recommend you turn on.

Now we can see the innards of the collection when debugging – YAY!

Leave a Comment

Setup OpenVPN Server with Streisand VPN on Ubuntu 16.04 LTS and connect with Windows Client

Let’s say that you need a run-of-the-mill VPN for your own purposes and you aren’t worried about using VPN-detecting services like Netflix then OpenVPN is a great option. Unfortunately it’s also a configuration nightmare; so lucky us then that the folks over at Streisand VPN have taken care of that.

Step 0 : Unattended Upgrades

The preparation step is to install unattended upgrades as an un-patched web-server is a really bad thing

sudo apt-get update && sudo apt-get upgrade
sudo apt-get install unattended-upgrades apt-listchanges

Now to ensure /etc/apt/apt.conf.d/20auto-upgrades exists

dpkg-reconfigure -plow unattended-upgrades

Lets give it a test run to ensure things aren’t configured wrong

sudo unattended-upgrade -d

Step 1 : Setup of OpenVPN through Streisand VPN

Streisand is a VPN setup script that automatically installs and configures many popular VPNs including OpenVPN and WireGuard

First we need to generate an SSH keypair for Streisand to use (back these up since they will be necessary to login from SSH after a reboot)

ssh-keygen -t rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

Now we need to install the dependencies and clone the github (note we don’t use the official github address here since there is a bug-fix we would like)

sudo add-apt-repository ppa:ansible/ansible-2.8
sudo apt-get update
sudo apt-get install ansible python
git clone https://github.com/akumaburn/streisand.git && cd streisand

Now execute the setup script, and be sure to enter your server’s IP address when prompted.

Also do NOT USE the default install, you can use all the default values in the custom install EXCEPT do not install wireguard and shadowsocks.

./streisand

At a certain point you may get an error about some tunnelblick issue, simply hit Ctrl +C and then C to continue

Once it is finished you will have your documentation available to you in the ~/streisand/generated-docs/tunnel.html file

Save it somewhere and open the file in a web-browser and follow the instructions

Step 2 : Configuration of OpenVPN

By default, OpenVPN is configured by the installation script to not allow multiple clients to use the same common-name (mentioned in the certificate), if you wish to change that you can by

sudo nano /etc/openvpn/server.conf

Find and un-comment this line so that you won’t need to generate a unique certificate for every VPN client

duplicate-cn

And hit Ctrl+o and Ctrl+x to save and exit

Now simply restart the OpenVPN server

sudo systemctl restart openvpn@server

Now you need to download the OpenVPN Connect Application and import the config file (you can pick any from the list on your website after following the tunnel.html instructions)

Now you should be able to connect

That’s it 🙂

Leave a Comment

Windows Tweaks and Fixes

This is a collection of scripts I regularly use to fix windows problems; some of them I have written myself and some I’ve gotten from others.

ClearEventViewerLog.ps1 – Clears the event viewer log that can get easily bloated over time

DisableMemoryCompression.ps1 – Disables Windows Memory Compression, which should improve application latency at the cost of a little extra memory

EnableLongPaths.ps1 – Enables Windows to deal with file paths that are longer than 255 characters

Fix_Wifi_Ping_Spikes.bat – Enables/Disables Windows WiFi Discovery; when it is off ping spikes are dramatically reduced, sometimes from 70ms down to 2ms

Regular_Maintenance.bat – Very powerful windows repair script that I wrote, does the usual DISM and SFC stuff, and also fixes the Windows Store, Start Menu and other Windows exclusive packages – This script can often bring Windows 10 back from the brink of death

Bring Back/Remove Open With Option on Context Menu.bat – This script will restore the “Open With” context dialog in the event that it has gone missing

SearchAndPackagesFix.ps1 – This script I wrote attempts to re-install all windows packages and also tries to fix the Windows start menu (It is used by Regular_Maintenance.bat)

Decrapify.ps1 – A powerful third-party script that lets you remove bloatware from Windows (be very careful what you select to remove as some packages are required by Windows)

You can download them all as a zip file here -> https://amireslampanah.com/Tweak_Scripts.zip

The password is: “amire”

(without the quotes)

Additionally I like to use BleachBit to clean the temp file mess created by Windows update and Wise Registry Cleaner(This link gives you a discount you can’t ordinary get) to clean the Windows Registry.

Discount on Advanced System Care 10 with Driver Booster Pro enter code 25OFF

Leave a Comment