Monday, June 5, 2017

vSphere ESXi backup strategies for lowering power consumption

For my home development servers I use the vSphere ESXi 6.5 hypervisor.


At home I do not need all the virtual machines on that server to be online 24/24 365/365, so this server is not always powered on. I need strategies to power up this server on needs.
That was achived by using the Wake on LAN (WoL).
I enable WOL on BIOS of that PC, so I can start the server whenever I need using the poweroff.exe utility (http://users.telenet.be/jbosman/applications.html) and a small batch file that simply launch poweroff.
Let's suppose my server has the IP 192.168.1.10 and mac 010203040506 the command below should start the server:
poweroff.exe wol -ip 192.168.1.10 -subnet 255.255.255.0 -mac 010203040506

Then I need something to backup virtual machines on that hypervisor, I build a bounch of script to perform the backup strategy I need.

But at first, let's talk about the hypervisor and the hardware used.
I was using ESXi since 5.0, this year I switch from 5.5 to 6.5.
My main server it's a HP Compaq dx2400 microtower powered by a Xeon E5440 and 8Gb RAM, the hypervisor is intalled on a 32Gb USB flash drive and hosts on a 1Tb HDD. It uses almost 60W.
This PC comes out with a Core 2 Due processor, but I've upgraded it to a Xeon E5440, due to the simplicity of the operation. A note about this upgrade, the Xeon processor I've used comes from a chinese seller already modded for the LGA 775 socket, also the latest BIOS for the motherboard of this PC (5.37 Rev.) does support the Xeon E5440.
I just need to install the new Xeon CPU, reset the BIOS, and power it up. Then I've enabled VT on bios.
For further information on this mod, search for LGA 771 to 775 adapter mod. To be sure my motherboard supports this CPU I've used the intelmicrocodelist.exe tool to look at the CPU supported by my motherboard BIOS.

I would like to backup each virtual machine on a remote disk. I've done this mounting a NFS disk on the ESXi.
The NFS service and disk is provided by a Netgear Stora NAS.
On that NAS i enabled NFS by kernel mode, If you've fot a Netgear Stora you can find further information here: http://www.openstora.com/wiki/index.php?title=Kernel_mode_NFS.
I use a Samba shared folder as main NFS entry point, in order to simply the backups download and look at all the files. My NFS entry in /etc/exports map the NFS user to the main stora user, that way by the share I can eventually delete files. On NFS this is authorized by the all_squash option. Let's suppose my store username is yourusername, the NFS folder is /home/yourusername/folder and yourusername id and group id are 10 and 11 (use the folling commands to get a username and group id on linux $id -u yourusername and $id -g yourusername).
My /etc/export entry looks like this:
/home/yourusername/folder 192.168.1.0/255.255.255.0(rw,sync,all_squash,anonuid=10,anongid=11,no_subtree_check)
As a side note, that Netgear Stora NAS is also modded to mount two disk wihout raid. I use one of the disk for PCs backup and the other one for ESXi backups.

Now, let's came to the backups.
I've chosen ghettoVCB script (https://github.com/lamw/ghettoVCB) as my backup engine because it's simple, opensource, and well documented.
ghettoVBS is is installed using instructions from the ghettoVCB page. This are the basic steps:
Download the ghettoVCB offline bundle to tmp folder and run
$esxcli software vib install -d /vghetto-ghettoVCB-offline-bundle.zip -f
Mount a NFS datastore on the hypervisor (ex backups01)
Make a ghettoVCB folder in the backup datastore (or other one)
$mkdir /vmfs/volumes/backups01/ghettovcb
Copy ghettoVCB.conf to the folder created
$cp /etc/ghettovcb/ghettoVCB.conf /vmfs/volumes/backups01/ghettovcb/ghettoVCB.conf
Edit ghettoVCB.conf at least set the backup folder
  VM_BACKUP_VOLUME=/vmfs/volumes/backups01
and the backup rotation count
  VM_BACKUP_ROTATION_COUNT=2

The backup strategy I've implemented is simple.
I run ghettoVCB each week to perform a full backup of each virtual machine, ghettoVCB handles the backups rotation. In order to lowering power consumption (remember that this server is not always on) I've implement this simple strategy, running a script at the the boot stage the hypervisor, This script:
  1. adds ghettoVCB crontab for day N of week time HH:MM+20min
  2. check if the hypervistor is switched on "next to" day N of week time HH:MM, if so
    • add a shutdown trigger file (the shutdown operation only runs if exists a shutdown trigger file)
    • add crontab for shutdown the server after H hours
An external device sends a WOL packet at day N of week time HH:MM-10min, if the hypervisor it is switched on at that time nothing happens, if not it will power up, schedule the ghettoVCB backup and eventually power down after the backups end.
The shutdown trigger file it is usefull to interrupt the shutdown procedure. Imagine that one want the server to stay on, he just has to delete the shutdown trigger file, available on the NFS Samba shared folder of backups.
This is all packed in the backuphelper.sh file you can find here https://gist.github.com/davidegironi/73f7dc8c65625d51b9a45e9fbe2f73c2
To use the backuphelper, you have to make a backuphelper folded
$mkdir /vmfs/volumes/backups01/backuphelper
Copy backuphelper.sh and esxidown.sh to the folder created
$cp /tmp/backuphelper.sh /vmfs/volumes/backups01/backuphelper/backuphelper.sh
$cp /tmp/esxidown.sh /vmfs/volumes/backups01/backuphelper/esxidown.sh
Edit backuphelper.sh setting the running parameters at top of file
Run backuphelper.sh at startup, edit file /etc/rc.local.d/local.sh adding commands below
  /vmfs/volumes/backups01/backuphelper/backuphelper.sh S

This backuphelper uses the esxidown script (https://github.com/sixdimensionalarray/esxidown) to gently shutdown the hypervisor.

The external device I'm using to send the WOL packet is a D-Link DSL-2750B/DSL-2751 rev D1 with OpenWrt installed, which I'm using as the main DHCP/DNS server in my home network. On this OpenWrt the etherwake package is installed, the etherwake command for WOL is set as a Scheduled Task.
Let's suppose my server mac is 010203040506 and I want to switch the server on on day 6 at 2:10 using the eth0.1 port, the command will look like this:
10 2 * * 6 /usr/bin/etherwake -D -i "eth0.1" "01:02:03:04:05:06"
Also, if you would like to make a backup, let's say the first saturday of month, the crontab schedule will look like:
10 2 1-7 *   *  [ "$(date '+%a')" = "Sat" ] && /usr/bin/etherwake -D -i "eth0.1" "01:02:03:04:05:06"

Another thing to take in consideration is the timezone, and time accuracy, all my devices have the NTP client enabled, and all the timezone are UTC.


Notes
  • read risk disclaimer
  • excuse my bad english

Monday, May 1, 2017

MQ gas sensor correlation function estimation by datasheet


The MQ series are cheap gas sensors.
They are resistive electro-chemical sensors.
These sensors have a heating element and a resistance that react to gases.
Below a MQ135 sensor opened. Inside it looks like that.



To read the sensor's resistance, we measure the voltage drop of the sensor.
The method we use to measure resistance on the micro is by using the ADC, and a pull-up (or pull-down) resistor.  Here http://electronics.stackexchange.com/a/70012 you can find a good explanation on the pull-down/up method on reading resistance.

Some people are asking me in my previous post about the MQ135 sensor, http://davidegironi.blogspot.it/2014/01/cheap-co2-meter-using-mq135-sensor-with.html how to obtain correlation function for other MQ series sensor, or other gas.

In this post I will explain a simple (to me) method of obtaining correlation function for any MQ series sensors that fit an exponential function.

An italian version of this article has been published in the may number of Elettronica In (http://www.elettronicain.it/) magazine.


As a reference, let's consider the MQ135 sensor (again).
We will look at the "sensitivity characteristics of the MQ-135" figure of the datasheet.


Notice that it's a log-log plot, this means that is has logarithmic scales on both axes.
Another thing to notice, is that the y-axis is for Rs/Ro, it will be much clearer if we impose ppm on the y-axis.
So as a first step, we will flip the axis.
As a reference, above we will see a flip axis example


Now, let's take a look at a power function with different exponents.


Let's try to plot those function on a log-log scale.


You may notice that an exponential function flipped is again an exponential function, with different coefficient of course.
As you may see, the power function with a negative exponent looks like the one we are searching for.
Summarizing, let's take the power function y=x^-2 and plot this on the log-log scale, with a 10 multiplier


We are searching for something like:

y=a*x^b

with b<0

Now, our purpose is to find out the value of the and b constants, starting from the datasheet figure.
We can do this by performin a nonlinear regression. Specifically we need a power regression.
First, we need to collect values from the datasheet figure.
WebPlotDigitizer is the tool we are going to use here: https://github.com/ankitrohatgi/WebPlotDigitizer. We can find few online tutorials that teach us how to use WePlotDigitalizer. I will try here to sum it up here:
1) extract the figure from the datasheet
2) load the figure on WebPlotDigitalizer
3) set axis, remember to se axis both as log
4) check out points from the curve of the gas you need to found
5) click on the view data button, and you get the points you need.

Now that we have found out the points of the curve from the datasheet, we need something to load with those points and perform a power regression.
You can use matlab or you favorite math software tools. There are also other online tools you can use to perform power regressions. Here I will use an R (ref. https://en.wikipedia.org/wiki/R_(programming_language)) script I've write.

You just have to load the points values you have found out, and launch this R script. You can also launch it on http://www.r-fiddle.org/.
The script will find out the function coefficients (a and b), and print out the function found next to the graph points, just to check the function's accuracy.

Below the Rs/Ro vs ppm function, as seen on the datasheet.


Then the flipped function we are searching for


Then, the plot of both functions on linear scale axes:




Now we have to estimate the Ro coefficient.
Ro estimation is quite simple
We just need to solve this equation for Ro
ppm=a*(Rs/Ro)^b
Ro=Rs* sqrt(a/ppm,b)=Rs*(a/ppm)^(1/b)
note: in my library here: http://davidegironi.blogspot.it/2014/01/cheap-co2-meter-using-mq135-sensor-with.html i simply Ro as Rs*sqrt(a/ppm,b) = Rs*exp(ln(a/ppm)/b), which is the same that Rs*(a/ppm)^b.

So given the function coefficients, we have to measure the resistance of the sensor at a know amount of ppm for the gas we are investigating.

Now, just substitute the found function coefficients (a and b), the resisteance we have read (Rs in ohm) at the amount of gas (ppm) to the estimation Ro function
Ro=Rs*(a/ppm)^b

One last step is to provide the Rs/Ro limits we are going to use to validate the input for our solving function ppm=a*(Rs/Ro)^b
We know that Rs/Ro=(ppm/a)^(1/b)
So
min[Rs/Ro]=(max[ppm]/a)^(1/b)
max[Rs/Ro]=(min[ppm]/a)^(1/b)
By the datasheet figure we have to select the max and min Rs/Ro and ppm points.
Notice that we get the max Rs/Ro value for the min ppm point value.

Here you can find the R script to estimate correlation function coefficients, compute Ro, min and max Rs/Ro (input: x/y axis limits, correlation function points, resistance of the sensor in ohm at a know amount of gas in ppm).
On gist: https://gist.github.com/davidegironi/b7be6b7cace6b475dd42c48c3e62fcf4

Above an implementation of the raw value to ppm conversion, sample sketch for Arduino:

//sensor input PIN
int mqInput = A1;
//pull-down resistor value
int mqR = 22000;
//rO sensor value
long rO = 41763;
//min value for Rs/Ro
float minRsRo = 0.358;
//max value for Rs/Ro
float maxRsRo = 2.428;
//sensor a coefficient value
float a = 116.6020682;
//sensor b coefficient value
float b = -2.769034857;

void setup() {
  pinMode(mqInput, INPUT);
  Serial.begin(9600);
}

void loop() {
  int adcRaw = analogRead(mqInput);
  long rS = ((1024.0 * mqR) / adcRaw) - mqR;
  Serial.print("Rs: ");
  Serial.println(rS);
  float rSrO = (float)rS / (float)rO;
  Serial.print("Rs/Ro: ");
  Serial.println(rSrO);
  if(rSrO < maxRsRo && rSrO > minRsRo) {
 float ppm = a * pow((float)rS / (float)rO, b);
 Serial.print("ppm: ");
 Serial.println(ppm);
  } else {
 Serial.println("Out of range.");
  }
  delay(1000);
}


Notes
  • read risk disclaimer
  • excuse my bad english

Sunday, April 2, 2017

An AVR ATmega R/C control signal reader - Method/01


The Radio Control (R/C or RC) is the method of controlling remote devices using radio frequencies.
Most of the time the RC signal uses RC servo signals.
Servo signal is a square PWM signal, usually with a period of 20ms and a duty time between 1ms and 2ms. More info on the wikipedia page here: https://en.wikipedia.org/wiki/Servo_control.


This library implements a method to read an RC signal and eventually map this to a 0..100 number.

It's developed on an ATmega8 running at 16Mhz.
It makes use of the external interrupt PIN and a timer to count the duty period time.
It's customizable to fit the user preferences. Configuration can be found in rcin1.h header file.
Theory of operation: each time a rising edge interrupt it is raised, a timer starts counting until the falling edge happens. Then the number of ticks counted between interrupts is transformed to us spent in duty time.
Input PIN is INT0.
The RC signal it is measured and then mapped to a speed value from 0 to 100.


Code
Notes
  • read risk disclaimer
  • excuse my bad english

Wednesday, March 1, 2017

An Arduino Knight Rider Rainbow dice that uses WS2812B RGB leds


This project implements a Knight Rider / Rainbow effect Random Selector.
It uses an Arduino UNO and a WS2812B RGB led strip.
A friend of mine needs a random selector for train scale model.
I've developed this using the Arduino framework, because he would like to modify the sketch "the Arduino way".
Unluckly I have not any picture or video of his build. So I've built a dice selector by using an old clock frame. I would like to thank my friend Matteo for cutting vinyls out for this project.


This project uses the Adafruit "NeoPixel library" (https://github.com/adafruit/Adafruit_NeoPixel).
For the Knight Rider Rainbow effect I took inspiration from the technobly "NeoPixel Knight Rider" project (https://github.com/technobly/NeoPixel-KnightRider).
The circuit for this project is really simple, there is just a Start switch connected, and a WS2812B led strip.



When the user press the start switch, the  Knight Rider / Rainbow selector starts, and a semi-randomic led is choses.
After a while, a Rainbow effect starts powering on all the leds.




Code

Notes
  • read risk disclaimer
  • excuse my bad english

Saturday, February 4, 2017

Duck DNS ESP8266 mini WiFi client


This is a Duck DNS ESP8266 mini WiFi client.


Duck DNS (https://www.duckdns.org/) is a free Dynamic DNS (DDNS) service.
It as a few client available for many operating system and a few routers.
A friend of mine runs a router that does not support any of the Duck DNS update mode. Instead of loading his operating system with the Duck DNS updater, or installing an openwrt router, I've implemented an ESP8266 client.


It is powered by USB, it can also be powered by the router USB port.
It's built on a pretty old ESP-01 board.
It has two led, one is the ESP-01 WiFi connection status embedded one, the other is connected to the GPIO2 port, and it's used for the DNS update status.


We need just a 3.3 voltage regulator to run this board using any 5V USB power output capable of 200mA.
The code is written using the Arduino IDE + ESP8266 core, to me that's the fastest and simple way to work with ESP8266.

The WiFiManager library (https://github.com/tzapu/WiFiManager) is used to facilitate the WiFi connection of this module.

Also it runs a web server that allow any user to setup the DuckDNS connection parameters.
User can set the DDNS domain name and token, and also the update interval and the module hostname, that way one can just forget the module IP and open the web configuration page by board name. The default board name is espduckdns001, so the web page address is http://espduckdns001


Code

Notes
  • read risk disclaimer
  • excuse my bad english


Tuesday, January 3, 2017

Mivar Samo UCM/591 vacuum tube radio repair


Mivar is an italian factory funded by Carlo Vichi in Milan, near to my house.
Some times ago I've found a 1960 vacuum tube radio, the Mivar UCM/591. It's really a nice looking device to me. Unluckly something in this radio is broken, so I've try to fix it. I'm not a vacuum tube expert, people from http://elettronialtramonto.forumfree.it/ helps me fixing this. Electrons at sunset, is a forum and a site devoted to beautifull but outdated technology.


WARNING! - The project described in these pages utilizes POTENTIALLY FATAL HIGH VOLTAGES. Do not attempt to build circuits presented on this site if you do not have the required experience and skills to work with such voltages. I assume no responsibility whatsoever for any damage caused by the usage of my circuits.

This radio has an annoying noise and whistle at low and high volume.
Before repairing a radio like this, keep in mind that it runs on high voltage, this device also is dangeros because of the power transformer type, that is an autotransformer. The case is connected to the main voltage grid, and no ground connection is provided. Autotransformer where usually installed on those cheap old radio. Pay attention to this is you are looking at an old tube device.
Below the schematics for this radio.




After checking the circuit, the first noticeable thing is the cathode electrolic cap on the power amp tube (the UL84) damaged.


I've changed it, but noise still exists. Checking things around UL84, I've noticed that the anode to ground cap (the 4k7 one) was missing. Likely this radio was repaired by someone that eventually forget that cap. Noise went down, but still is there. The next step was checking the cathode resistor, it was a 220ohm by color codes (even if the circuit I've found online list it as a 150ohm one), but it meausre more than 500ohm, I've replaced it. At this stage noise was almost gone, voltage on the UL84 grid is now 12.5V, but there's still an high whistle on low volume. Checking all the paper caps I've found that none of them is in range. Some are up to 300% out of their nominal value. I've to replace all (almost a dozen).


Paper cap where replaced by disk ceramic caps, cause i do not have old stock paper caps, electrolitics cap where replaced by new one. I've try to install caps from old device I've here in my junk box, just to mantain the old look of this device. Below a picture of the defunct components.


I've tested it, voltages are next to the schematics one, now all is working.
So, thanks again to people from elettronialtramonto that helps me.


The repaired board looks like the picture below.


Notes
  • read risk disclaimer
  • excuse my bad english