Category: Unix/Linux

Spinning up a bunch of virtual desktops in Amazon WorkSpaces (videos)

This was a pretty fun project that I had gotten so I figured I would share the experience. There are multiple use cases for virtual desktops. In our case, it’s ephemeral – only need them for a few days for a class so that all students can share the same experience without the need for anything but a web browser. They can probably get a better experience with the PCoIP client, but it could be against some company policies. Most companies will allow HTTPs out, so we figured this would be the easiest way.

The way Amazon WorkSpaces works is that each desktop is assigned to a single user and the users sit in the directory service. The service I’m using is the Simple AD (Samba 4) as I had no need for a huge directory. To create the users, we will just need a UID (sAMAccountName in AD) and a password if using the API to create the desktops. If using the Amazon portal to create the desktops, you’ll need the first and last name and an email address as well. You can easily import a CSV file with this information, but for the sake of simplicity, I just use a generic account name and numbers.

After creating the directory and starting up a single desktop, I went to the “Programs” in the Control Panel and “Turn Windows Features on and off” and “Features” to install the “AD DS and AD LDS Tools”. More information on the RSAT tools is available here: https://wiki.samba.org/index.php/Installing_RSAT 

Here’s a short video on how to do it:

Once the RSAT tools are installed, the “dsadd” command will be available to add users. This is the script I’m using that asks for the users and then creates the users:

echo off
set /p users=Number of users to create:
echo "Creating %users% students"
set count=0
:createusers
    set /a count+=1
    echo creating student%count%
    dsadd user "cn=student%count%,cn=users,dc=corp,dc=amazonworkspaces,dc=com" -samid student%count% -pwd Student%count%
    if "%count%"=="%users%" goto done
    goto createusers
:done

The script will create users with the username student# with passwords Student# – the capital “S” is just for password complexity.

After creating the users, we can go and create the desktops. To do this, I used awscli. On a Mac or Linux system, it can be easily installed running “easy_install awscli”. After installation, there will be a config and credentials file that should be configured in the .aws directory in your home directory. Once that’s set, you can check to see what workspaces you have by running “aws workspaces describe-workspaces” – that gives you an idea of what your workspaces look like. The minimal template I’m using for workspaces looks like this:

{
 "Workspaces" : [
 {
   "DirectoryId" : "d-9267258c77",
   "UserName" : "%username%",
   "WorkspaceProperties": {
   "RunningMode": "AUTO_STOP"
 },
   "BundleId" : "wsb-gw81fmq2p"
 }
 ]
}

The DirectoryId is the directory service where the users are housed, I’ll be replacing the %username% with student#, and I added RunningMode just to save on costs – they’ll automatically suspend after an hour of idling. It takes about 90s to spin back up if they suspend. The BundleId is the VM that you want to provision. This one is the customized one for our classroom.

With the template in place, we’re ready to run the script:

#!/bin/bash

echo "Number of Desktops to Create [20?]"
read desktops
echo $desktops
COUNTER=0
         while [  $COUNTER -lt $desktops ]; do
             let COUNTER=COUNTER+1
             echo Creating Desktop number $COUNTER
        sed "s/%username%/student$COUNTER/g" create-workspaces.json > /tmp/student$COUNTER.json
        aws workspaces create-workspaces --cli-input-json file:///tmp/student$COUNTER.json
         done
echo Created $desktops Desktops.

You can remove the temporary files in /tmp afterwards.

Here’s a short video of the scripts in action.

Have fun with your desktops!

AIX notes … ipfilter, unzip, zlib, openssh, openssl

I had the privilege of experiencing AIX for the very first time this week. Hopefully this can save someone else time.

Some packages that aren’t installed by default that you might want include openssl, openssh, unzip, zlib, and IPFilter.

I would probably start with openssl/openssh. In AIX 7.2, you can do it in the OS installer. To do it outside of the installer, keep the installation cd in and run the following commands:

mount -V cdrfs -o ro /dev/cd0 /mnt
cd /mnt/usr/sys/inst.images/
installp -ac -Y -d . openssh.base openssl.base openssl.man.en_US openssh.man.en_US
lssrc -s sshd
umount /mnt

The default partitions aren’t big enough! Fortunately, it’s very easy to extend the partitions. You can do so with the following commands:

chfs -a size=+4G /opt
chfs -a size=+4G /var
chfs -a size=+4G /home
chfs -a size=+4G /usr
chfs -a size=+2G /tmp
chfs -a size=+4G /admin

Installing 3rd party software:

You can download unzip from:  https://public.dhe.ibm.com/aix/freeSoftware/aixtoolbox/RPMS/ppc/unzip/unzip-6.0-3.aix6.1.ppc.rpm. You can install it with “rpm -i” just like in Linux. Another open for unzipping files without unzip is using jar. You can run “jar -xvf” on a file and it can unzip it as well.

If you need the zlib library, you can get it from:  https://public.dhe.ibm.com/aix/freeSoftware/aixtoolbox/RPMS/ppc/zlib/zlib-1.2.11-1.aix6.1.ppc.rpm.

You can install IPFilter from https://www-01.ibm.com/marketing/iwm/iwm/web/reg/pick.do?source=aixbp. It will require a login, but not a serial number. Just create a login and download. Installing IPFilter is a little different. It installs like an AIX package, with installp. Unzip the contents of the IPFilter_Fileset.zip and go into the IPFilter_Fileset directory and run the following commands:

inutoc .
installp -ac -gXY -d. ipfl

Help! SSH attack on Ubuntu 14.04.2 LTS – use sshdfilter with ufw

If you’ve got a *Nix machine on the public Internet, you will experience this at one time or another. If you run sshd on its’ known port 22, some script kiddie out there will attack it. With that, you’ll see that you’ll have a bunch of connections that probably and hopefully will never succeed. I saw these messages in my /var/log/auth.log:

Nov 15 06:44:26 chunli sshd[20634]: Failed password for root from 43.229.53.13 port 41751 ssh2
Nov 15 06:44:26 chunli sshd[20636]: Failed password for root from 43.229.53.13 port 41921 ssh2
Nov 15 06:44:26 chunli sshd[20638]: Failed password for root from 43.229.53.13 port 42948 ssh2
Nov 15 06:44:26 chunli sshd[20546]: message repeated 2 times: [ Failed password for root from 43.229.53.13 port 27586 ssh2]

I’ve experienced this before and to alleviate this problem, I used sshdfilter in the past. I know that there are others out there like sshblack that can do blacklisting of attackers or one of the easiest ways to alleviate this problem is just to run ‘sudo ufw limit ssh’. This is a way to block them from coming back for a while.

To get started, first stop SSH:

sudo initctl stop ssh

Then move /etc/init/ssh.conf out. We’ll start SSH with sshdfilter instead.

You’ll get a message in the auth.log here:
auth.log:Nov 16 19:54:36 chunli sshd[1150]: Missing privilege separation directory: /var/run/sshd

Because of this, we’ll just add a line in the /etc/init.d/sshdfilter file like this:

start() {
 echo -n $"Starting sshdfilter: "
 export PATH=$PATH:/usr/local/sbin
 mkdir /var/run/sshd
 sshdfilter
 RETVAL=$?
 return $RETVAL

In /etc/sshdfilterrc, I changed the chain from:

#chain=’SSHD’ to

chain='ufw-reject-input'

Under the “Add a block rule” section, I changed some iptables commands to ufw commands. You can see the comments ones here:

#firewalladd='iptables -A $chain -p tcp -s $ip --dport 22 -j DROP'
#firewalladd='iptables -A $chain -p tcp -s $ip --dport 22 -j DROP'
firewalladd='ufw insert 1 reject proto tcp from $ip to any port 22'
# Delete a block rule:
firewalldel='ufw delete reject proto tcp from $ip to any port 22'
#firewalldel='iptables -D $chain -p tcp -s $ip --dport 22 -j DROP'

That’s about it! With that, my machine started to grow a big list of IP addresses to block from failed logins.

 

DNS server not resolving external addresses – bind9 Ubuntu 16.

I ran into issues with Bind 9 on Ubuntu 16 very recently. I use an internal caching DNS server for a few reasons – try to protect myself from malware, log DNS requests on my network to have an idea of where all hosts are going – mainly searching for malware or bad websites, block some domains i.e. ad blocking and some others that I might not want to visit, even by accident, etc. Suddenly at some point, I could no longer resolve any addresses that were not in my zones. My forwarders were no longer working, etc. It turned out to be:

dnssec-validation auto;

This line is default in Ubuntu 16’s Bind 9. Why it worked before, I don’t know. I changed it to

dnssec-validation no;

Then everything magically started working again. Hope this saves someone else’s time. 🙂

Protect your home network using TomatoUSB – how to only allow only HTTP/S out!

While we continue to see the WannaCry and other malware around, I thought I would secure my own network. Since I allow visitors onto their networks, I figured I would configure all new DHCP’d hosts to access the Internet only via HTTP and HTTPs and not allow them to use any DNS servers other than OpenDNS. Here’s how to do it:

The first thing I did was create an access restriction. I did this just to see what chain would be created and I would put subsequent rules into that chain.

access restriction screenshot

The previous screenshot created this chain:

Chain rdev07 (1 references)
target prot opt source destination
DROP all -- 192.168.0.15 anywhere

With this chain, I can add additional rules. The first thing I want to do is allow only DNS access to OpenDNS servers and none other. For this, I would run the following commands:

iptables -A rdev07 -4 -p tcp -s 192.168.0.0/24 -d 208.67.222.222/32 --dport 53 -j ACCEPT
iptables -A rdev07 -4 -p udp -s 192.168.0.0/24 -d 208.67.222.222/32 --dport 53 -j ACCEPT
iptables -A rdev07 -4 -p tcp -s 192.168.0.0/24 -d 208.67.220.220/32 --dport 53 -j ACCEPT
iptables -A rdev07 -4 -p udp -s 192.168.0.0/24 -d 208.67.220.220/32 --dport 53 -j ACCEPT
iptables -A rdev07 -4 -p tcp -s 192.168.0.0/24 -d 0.0.0.0/0/0 --dport 53 -j REJECT
iptables -A rdev07 -4 -p udp -s 192.168.0.0/24 -d 0.0.0.0/0/0 --dport 53 -j REJECT

These rules basically allow DNS queries from my network to the 2 OpenDNS servers. The last 2 rules mean that no other DNS servers outside of those 2 servers can be queried. The reason I do this is because there is some malware out there that will change the DNS servers to query on Windows, effectively overriding the DHCP setting. An alternative to this would be to configure Tomato to intercept DNS requests, but I would rather do it this way.

I added the following rules because I had noticed for some reason that some connections coming back from OpenDNS were dropped. I think they’re optional, but I put them in.

iptables -A rdev07 -4 -p tcp -s 208.67.222.222/32 -d 192.168.0.0/24 --sport 53 -j ACCEPT
iptables -A rdev07 -4 -p udp -s 208.67.222.222/32 -d 192.168.0.0/24 --sport 53 -j ACCEPT
iptables -A rdev07 -4 -p tcp -s 208.67.222.222/32 -d 192.168.0.0/24 --sport 53 -j ACCEPT
iptables -A rdev07 -4 -p udp -s 208.67.222.222/32 -d 192.168.0.0/24 --sport 53 -j ACCEPT

Next, I go to create my whitelist – this would be my iPhone, iPad, android, etc – any hosts that I trust. I’m going to allow these host to go out to any host with TCP and UDP.

 

iptables -A rdev07 -4 -p tcp -s 192.168.0.3/32 -d 0.0.0.0/0 -j ACCEPT
iptables -A rdev07 -4 -p tcp -s 192.168.0.11/32 -d 0.0.0.0/0 -j ACCEPT
iptables -A rdev07 -4 -p tcp -s 192.168.0.31/32 -d 0.0.0.0/0 -j ACCEPT
iptables -A rdev07 -4 -p udp -s 192.168.0.3/32 -d 0.0.0.0/0 -j ACCEPT
iptables -A rdev07 -4 -p udp -s 192.168.0.11/32 -d 0.0.0.0/0 -j ACCEPT
iptables -A rdev07 -4 -p udp -s 192.168.0.31/32 -d 0.0.0.0/0 -j ACCEPT
I know that they can still get viruses. I hope they don’t. They can only use OpenDNS for DNS services, but they can access basically anything outside on any port.
Lastly, I configure the rules to allow only HTTP and HTTPs out.
iptables -A rdev07 -4 -p tcp -s 192.168.0.0/24 -d 0.0.0.0/0 --dport 80 -j ACCEPT
iptables -A rdev07 -4 -p tcp -s 192.168.0.0/24 -d 0.0.0.0/0 --dport 443 -j ACCEPT
iptables -A rdev07 -4 -p all -s 192.168.0.0/24 -d 0.0.0.0/0 -j DROP
With this, anyone else on the network can connect to port 80 and 443 of any host on the Internet. Then, any traffic going out to any other port is dropped.
After testing all commands and seeing that they worked for me, I put them all into Administration/Scripts/Firewall.
Inserting custom firewall rules
Have fun and be safe! Please post any comments below.

How to send tweets at random times with a *NIX CLI …

Twitter CLI is a wonderful tool for automatically sending tweets. Very simply, after installation, you just need to run ‘t update “message”‘ and it would tweet your message. I wanted to do some tweeting randomly via cron, but since cron runs regularly at an interval, it’s not very random. Just adding a couple of lines to the script will make it random.

Here’s how to do it.

First, download and install Twitter CLI from here: https://github.com/sferik/t

Afterwards, make a copy of the t script and call it what you want. I call it randomt:

cp /usr/local/bin/t /usr/local/bin/randomt

Finally, just make a change to the script. Here, I’m setting a variable called time and making it a random number between 1 and 1000. From there, I’m multiplying it by 60 to convert it to minutes.

#!/usr/bin/ruby2.0
#
# This file was generated by RubyGems.
#
# The application 't' is installed as part of a gem, and
# this file is here to facilitate running it.
#

require 'rubygems'

version = ">= 0"

time = 1 + rand(1000) * 60
sleep(time)

if ARGV.first
 str = ARGV.first
 str = str.dup.force_encoding("BINARY") if str.respond_to? :force_encoding
 if str =~ /\A_(.*)_\z/
 version = $1
 ARGV.shift
 end
end

gem 't', version
load Gem.bin_path('t', 't', version)

Now, my tweets go out at some random time between 1 and 1000 minutes later. Thanks for reading!

 

 

esxtop shows a bunch of stuff … what’s up with this!?

It’s been a while since I’ve had to run esxtop, but when I did recently, I got a bunch of stuff on my screen like this:

o termcap entry for a `xterm-256color’ terminal”(PDH-CSV 4.0) (UTC)(0)”,”\\localhost.shocknetwork.com\Memory\Memory Overcommit (1 Minute Avg)”,”\\localhost.shocknetwork.com\Memory\Memory Overcommit (5 Minute Avg)”,”\\localhost.shocknetwork.com\Memory\Memory Overcommit (15 Minute Avg)”,”\\localhost.shocknetwork.com\Physical Cpu Load\Cpu Load (1 Minute Avg)”,”\\localhost.shocknetwork.com\Physical Cpu Load\Cpu Load (5 Minute Avg)”,”\\localhost.shocknetwork.com\Physical Cpu Load\Cpu Load (15 Minute Avg)”,”\\localhost.shocknetwork.com\Physical Cpu(0)\% Processor Time”,”\\localhost.shocknetwork.com\Physical Cpu(1)\% Processor Time”,”\\localhost.shocknetwork.com\Physical Cpu(2)\% Processor Time”,”\\localhost.shocknetwork.com\Physical Cpu(3)\% Processor Time”,”\\localhost.shocknetwork.com\Physical Cpu(_Total)\% Processor Time”,”\\localhost.shocknetwork.com\Physical Cpu(0)\% Util Time”,”\\localhost.shocknetwork.com\Physical Cpu(1)\% Util Time”,”\\localhost.shocknetwork.com\Physical Cpu(2)\% Util Time”,”\\localhost.shocknetwork.com\Physical Cpu(3)\% Util Time”,”\\localhost.shocknetwork.com\Physical Cpu(_Total)\% Util Time”,”\\localhost.shocknetwork.com\Physical Cpu(0)\% Core Util Time”,”\\localhost.shocknetwork.com\Physical Cpu(1)\% Core Util Time”,”\\localhost.shocknetwork.com\Physical Cpu(2)\% Core Util Time”,”\\localhost.shocknetwork.com\Physical Cpu(3)\% Core Util Time”,”\\localhost.shocknetwork.com\Physical Cpu(_Total)\% Core Util Time”,”\\localhost.shocknetwork.com\Memory\Machine MBytes”,”\\localhost.shocknetwork.com\Memory\Kernel MBytes”,”\\localhost.shocknetwork.com\Memory\NonKernel MBytes”,”\\localhost.shocknetwork.com\Memory\Free MBytes”,”\\localhost.shocknetwork.com\Memory\Kernel Managed MBytes”,”\\localhost.shocknetwork.com\Memory\Kernel MinFree MBytes”,”\\localhost.shocknetwork.com\Memory\Kernel Reserved MBytes”,”\\localhost.shocknetwork.com\Memory\Kernel Unreserved MBytes”,”\\localhost.shocknetwork.com\Memory\Kernel State”,”\\localhost.shocknetwork.com\Memory\PShare Shared MBytes”,”\\localhost.shocknetwork.com\Memory\PShare Common MBytes”,”\\localhost.shocknetwork.com\Memory\PShare Savings MBytes”,”\\localhost.shocknetwork.com\Memory\Swap Used MBytes”,”\\localhost.shocknetwork.com\Memory\Swap Target MBytes”,”\\localhost.shocknetwork.com\Memory\Swap MBytes Read/sec”,”\\localhost.shocknetwork.com\Memory\Swap MBytes Write/sec”,”\\localhost.shocknetwork.com\Memory\Total Compressed MBytes”,”\\localhost.shocknetwork.com\Memory\Total Saved By Compression MBytes”,”\\localhost.shocknetwork.com\Memory\Memctl Current MBytes”,”\\localhost.shocknetwork.com\Memory\Memctl Target MBytes”,”\\localhost.shocknetwork.com\Memory\Memctl Max MBytes”,”\\localhost.shocknetwork.com\Power\Power Usage Now Watts”,”\\localhost.shocknetwork.com\Power\Power Usage Cap Watts”,”\\localhost.shocknetwork.com\Power(0)\P-State MHZ”,”\\localhost.shocknetwork.com\Power(1)\P-State MHZ”,”\\localhost.shocknetwork.com\Power(2)\P-State MHZ”,”\\localhost.shocknetwork.com\Power(3)\P-State MHZ”,”\\localhost.shocknetwork.com\Power(4)\P-State MHZ”,”\\localhost.shocknetwork.com\Power(5)\P-State MHZ”,”\\localhost.shocknetwork.com\Power(6)\P-State MHZ”,”\\localhost.shocknetwork.com\Power(7)\P-State MHZ”,”\\localhost.shocknetwork.com\Power(8)\P-State MHZ”,”\\localhost.shocknetwork.com\Power(9)\P-State MHZ”,”\\localhost.shocknetwork.com\Power(10)\P-State MHZ”,”\\localhost.shocknetwork.com\Power(11)\P-State MHZ”,”\\localhost.shocknetwork.com\Power(12)\P-State MHZ”,”\\localhost.shocknetwork.com\Power(13)\P-State MHZ”,”\\localhost.shocknetwork.com\VSAN\Enabled?”,”\\localhost.shocknetwork.com\Group Cpu(1:idle)\Members”,”\\localhost.shocknetwork.com\Group Cpu(1:idle)\% Used”,”\\localhost.shocknetwork.com\Group Cpu(1:idle)\% Run”,”\\localhost.shocknetwork.com\Group Cpu(1:idle)\% System”,”\\localhost.shocknetwork.com\Group Cpu(1:idle)\% Wait”,”\\localhost.shocknetwork.com\Group Cpu(1:idle)\% Ready”,”\\localhost.shocknetwork.com\Group Cpu(1:idle)\% Idle”,”\\localhost.shocknetwork.com\Group Cpu(1:idle)\% Overlap”,”\\localhost.shocknetwork.com\Group Cpu(1:idle)\% CoStop”,”\\localhost.shocknetwork.com\Group Cpu(1:idle)\% Max Limited”,”\\localhost.shocknetwork.com\Group Cpu(1:idle)\% Swap Wait”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32769:idle1)\% Used”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32769:idle1)\% Run”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32769:idle1)\% System”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32769:idle1)\% Wait”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32769:idle1)\% VmWait”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32769:idle1)\% Ready”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32769:idle1)\% Idle”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32769:idle1)\% Overlap”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32769:idle1)\% CoStop”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32769:idle1)\% Max Limited”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32769:idle1)\% Swap Wait”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32770:idle2)\% Used”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32770:idle2)\% Run”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32770:idle2)\% System”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32770:idle2)\% Wait”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32770:idle2)\% VmWait”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32770:idle2)\% Ready”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32770:idle2)\% Idle”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32770:idle2)\% Overlap”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32770:idle2)\% CoStop”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32770:idle2)\% Max Limited”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32770:idle2)\% Swap Wait”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32771:idle3)\% Used”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32771:idle3)\% Run”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32771:idle3)\% System”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32771:idle3)\% Wait”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32771:idle3)\% VmWait”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32771:idle3)\% Ready”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32771:idle3)\% Idle”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32771:idle3)\% Overlap”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32771:idle3)\% CoStop”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32771:idle3)\% Max Limited”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32771:idle3)\% Swap Wait”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32778:idle0)\% Used”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32778:idle0)\% Run”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32778:idle0)\% System”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32778:idle0)\% Wait”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32778:idle0)\% VmWait”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32778:idle0)\% Ready”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32778:idle0)\% Idle”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32778:idle0)\% Overlap”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32778:idle0)\% CoStop”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32778:idle0)\% Max Limited”,”\\localhost.shocknetwork.com\Vcpu(1:idle:32778:idle0)\% Swap Wait”,”\\localhost.shocknetwork.com\Group Cpu(2:system)\Members”,”\\localhost.shocknetwork.com\Group Cpu(2:system)\% Used”,”\\localhost.shocknetwork.com\Group Cpu(2:system)\% Run”,”\\localhost.shocknetwork.com\Group Cpu(2:system)\% System”,”\\localhost.shocknetwork.com\Group Cpu(2:system)\% Wait”,”\\localhost.shocknetwork.com\Group Cpu(2:system)\% Ready”,”\\localhost.shocknetwork.com\Group Cpu(2:system)\% Idle”,”\\localhost.shocknetwork.com\Group Cpu(2:system)\% Overlap”,”\\localhost.shocknetwork.com\Group Cpu(2:system)\% CoStop”,”\\localhost.shocknetwork.com\Group Cpu(2:system)\% Max Limited”,”\\localhost.shocknetwork.com\Group Cpu(2:system)\% Swap Wait”,”\\localhost.shocknetwork.com\Vcpu(2:system:32772:SVGAConsole)\% Used”,”\\localhost.shocknetwork.com\Vcpu(2:system:32772:SVGAConsole)\% Run”,”\\localhost.shocknetwork.com\Vcpu(2:system:32772:SVGAConsole)\% System”,”\\localhost.shocknetwork.com\Vcpu(2:system:32772:SVGAConsole)\% Wait”,”\\localhost.shocknetwork.com\Vcpu(2:system:32772:SVGAConsole)\% VmWait”,”\\localhost.shocknetwork.com\Vcpu(2:system:32772:SVGAConsole)\% Ready”,”\\localhost.shocknetwork.com\Vcpu(2:system:32772:SVGAConsole)\% Idle”,”\\localhost.shocknetwork.com\Vcpu(2:system:32772:SVGAConsole)\% Overlap”,”\\localhost.shocknetwork.com\Vcpu(2:system:32772:SVGAConsole)\% CoStop”,”\\localhost.shocknetwork.com\Vcpu(2:system:32772:SVGAConsole)\% Max Limited”,”\\localhost.shocknetwork.com\Vcpu(2:system:32772:SVGAConsole)\% Swap Wait”,”\\localhost.shocknetwork.com\Vcpu(2:system:32773:debugtermlivedump)\% Used”,”\\localhost.shocknetwork.com\Vcpu(2:system:32773:debugtermlivedump)\% Run”,”\\localhost.shocknetwork.com\Vcpu(2:system:32773:debugtermlivedump)\% System”,”\\localhost.shocknetwork.com\Vcpu(2:system:32773:debugtermlivedump)\% Wait”,”\\localhost.shocknetwork.com\Vcpu(2:system:32773:debugtermlivedump)\% VmWait”,”\\localhost.shocknetwork.com\Vcpu(2:system:32773:debugtermlivedump)\% Ready”,”\\localhost.shocknetwork.com\Vcpu(2:system:32773:debugtermlivedump)\% Idle”,”\\localhost.shocknetwork.com\Vcpu(2:system:32773:debugtermlivedump)\% Overlap”,”\\localhost.shocknetwork.com\Vcpu(2:system:32773:debugtermlivedump)\% CoStop”,”\\localhost.shocknetwork.com\Vcpu(2:system:32773:debugtermlivedump)\% Max Limited”,”\\localhost.shocknetwork.com\Vcpu(2:system:32773:debugtermlivedump)\% Swap Wait”,”\\localhost.shocknetwork.com\Vcpu(2:system:32774:logSysAlert)\% Used”,”\\localhost.shocknetwork.com\Vcpu(2:system:32774:logSysAlert)\% Run”,”\\localhost.shocknetwork.com\Vcpu(2:system:32774:logSysAlert)\% System”,”\\localhost.shocknetwork.com\Vcpu(2:system:32774:logSysAlert)\% Wait”,”\\localhost.shocknetwork.com\Vcpu(2:system:32774:logSysAlert)\% VmWait”,”\\localhost.shocknetwork.com\Vcpu(2:system:32774:logSysAlert)\% Ready”,”\\localhost.shocknetwork.com\Vcpu(2:system:32774:logSysAlert)\% Idle”,”\\localhost.shocknetwork.com\Vcpu(2:system:32774:logSysAlert)\% Overlap”,”\\localhost.shocknetwork.com\Vcpu(2:system:32774:logSysAlert)\% CoStop”,”\\localhost.shocknetwork.com\Vcpu(2:system:32774:logSysAlert)\% Max Limited”,”\\localhost.shocknetwork.com\Vcpu(2:system:32774:logSysAlert)\% Swap Wait”,”\\localhost.shocknetwork.com\Vcpu(2:system:32775:serialLogger)\% Used”,”\\localhost.shocknetwork.com\Vcpu(2:system:32775:serialLogger)\% Run”,”\\localhost.shocknetwork.com\Vcpu(2:system:32775:serialLogger)\% System”,”\\localhost.shocknetwork.com\Vcpu(2:system:32775:serialLogger)\% Wait”,”\\localhost.shocknetwork.com\Vcpu(2:system:32775:serialLogger)\% VmWait”,”\\localhost.shocknetwork.com\Vcpu(2:system:32775:serialLogger)\% Ready”,”\\localhost.shocknetwork.com\Vcpu(2:system:32775:serialLogger)\% Idle”,”\\localhost.shocknetwork.com\Vcpu(2:system:32775:serialLogger)\% Overlap”,”\\localhost.shocknetwork.com\Vcpu(2:system:32775:serialLogger)\% CoStop”,”\\localhost.shocknetwork.com\Vcpu(2:system:32775:serialLogger)\% Max Limited”,”\\localhost.shocknetwork.com\Vcpu(2:system:32775:serialLogger)\% Swap Wait”,”\\localhost.shocknetwork.com\Vcpu(2:system:32776:tlbflushcount)\% Used”,”\\localhost.shocknetwork.com\Vcpu(2:system:32776:tlbflushcount)\% Run”,”\\localhost.shocknetwork.com\Vcpu(2:system:32776:tlbflushcount)\% System”,”\\localhost.shocknetwork.com\Vcpu(2:system:32776:tlbflushcount)\% Wait”,”\\localhost.shocknetwork.com\Vcpu(2:system:32776:tlbflushcount)\% VmWait”,”\\localhost.shocknetwork.com\Vcpu(2:system:32776:tlbflushcount)\% Ready

Continue reading “esxtop shows a bunch of stuff … what’s up with this!?”

SSH without a password using keys troubleshooting – use RSA instead?

I used a procedure from this: http://sshkeychain.sourceforge.net/mirrors/SSH-with-Keys-HOWTO/SSH-with-Keys-HOWTO-4.html

Basically, you just run ssh-keygen to generate a public key and put it on the server’s ~/.ssh/authorized_keys file and that’s about it. The issue I can into was that the server did not accept DSA keys, but accepted RSA keys.

If you suspect that you’re running into the same problem, you can add the “-v” argument to your ssh command and confirm. You should see something like this:
debug1: Skipping ssh-dss key /Users/altonyu/.ssh/id_dsa – not in PubkeyAcceptedKeyTypes

If that’s the case, try using rsa keys. You can generate the RSA key by running:

ssh-keygen -t rsa

Copy the key over to the server’s authorized_keys file and you should get in immediately!

Linux self-service firewall with Apache, Perl, IPtables, and UFW

I was recently in a situation where I was offering DNS service to some people. There just happened to be some records that were different from their ISPs DNS servers so I set up the server on my public IP address. I did not want to offer DNS to the world because last time I tried that, I got queries from all kinds of places for all kinds of records. I was initially opening up IP tables when people asked for the service and give me their IP address. After getting about 10 texts, I quickly got tired of collecting the IP addresses, so I made a webpage and with the perl script to write them to a list. With that, I would have a cron jobs go through the list and use UFW to update the IPTables to allow them access.

Here are the files inside of the directory where I’m creating the list.
dnsauth.tar

There’s a simple index.html file in the directory. It’s basically a form that asks for:
Name – who the person is. duh!
IP address – I want them to enter the IP address they want to authorize just in case they’re submitting someone else’s IP address.
Password – I don’t want just anyone to come in and get access to my DNS server.

The addip.cgi basically just writes all of those inputs and the IP address they’re coming in from into /tmp/iplist.txt in CSV format. I record the IP address they’re coming in from $ENV{‘REMOTE_ADDR’} just in case I get abuse or something.

The root user then has a cron job that runs through the iplist.txt file every 10 minutes. Here’s my file:

#!/bin/sh
if [ -f /tmp/iplist.txt ]; then
DATE=$(date +%Y%m%d)
cp /tmp/iplist.txt /home/alton/dnsservice/iplist.txt.$DATE
for i in `grep rice /tmp/iplist.txt | cut -f1 -d','`; do /usr/sbin/ufw insert 1 allow proto udp from $i to any port 53; done
grep rice /tmp/iplist.txt >> /home/alton/dnsservice/authorized_dns_ips.txt
grep -v rice /tmp/iplist.txt >> /home/alton/dnsservice/cheaters.txt
rm -rf /tmp/iplist.txt
sync
fi

Obviously, rice was my password. I just looped through the file and authorized anyone that used the right password. I also logged anyone that used the wrong password in /home/alton/dnsservice/cheaters.txt.

Hope this was useful! I welcome any comments. Obviously, this was quick and dirty. I’m sure there is a more secure way of doing this, but this is what came easy to me. Would love to hear your thoughts!

How to find and online all of your SteelFusion Core LUNs on a NetApp filer

Recently, I offlined a bunch of LUNs that had belonged to a SteelFusion Core in the lab that I had forgotten about. Needless to say, I had some unhappy users. The good news though is that I was able to get the LUNs back up and connected to the Core within minutes. This is how I did it.

The first thing I needed to do was find out which LUNs the Core was using. I did this by logging into the Core via SSH and running the following commands:

enable
conf t
terminal length 0
show storage luns iscsi

I output this to a file /tmp/core30luns.txt. An entry looks like this:

Total LUNs: 9
Locally Assigned Serial: P3PdB/-GFigd
Configuration status : Ready
Alias : avamar_restore
LUN Size : 150.00 GB
LUN Type : iscsi
Online : yes
IOPs acceleration : Enabled
Failover Enabled : yes
Prefetch : Enabled
Edge mapping : pod3-3100b
Target mapping : iqn.2003-10.com.riverbed:oh1mt0017065c.000
Origin portal : 10.33.192.174, 10.33.192.175
Origin target : iqn.1992-08.com.netapp:sn.135037602
Backend session status : Connected
Use iSCSI Reservation : Yes
LUN Edge data session : Connected
Client type : other
Original LUN vendor : NetApp
Original LUN serial : P3PdB/-GFigd
Pinned : no
Prepop : Disabled
Smart prepop : Enabled
Prepop status : N/A
MPIO policy : roundrobin
iSCSI Reservation status : LUN reserved

Prepop schedules:
Mapped igroups:
all

Mapped initiators:

The next thing was to find out what LUNs are on the NetApp to do some matching. You can do that by running this command:

lun show -v

I output this to a file /tmp/netapp_luns.txt. An entry looks like this:

/vol/NewYork_rvbd_d_e7cc5c29_f400_4c52_b1d4_f87da1b62652_1451278801/lun_RDM 10g (10737418240) (r/w, offline)
Serial#: P3PdB/9ytT31
Share: none
Space Reservation: disabled
Multiprotocol Type: vmware

Now with the 2 files, I could do some matching. I first want to extract the serial numbers from the LUNs. I do this by running:

grep serial /tmp/core30luns.txt | cut -f2 -d: > /tmp/core30lunlist.txt

From that, I would just get a list of serial numbers like this:

P3PdB/-GFigd

Next, I will loop through my list of LUNs to find the volumes I will need to put back online. I do this by running:

for i in `cat /tmp/core30lunlist.txt`; do grep -2 $i /tmp/netapp_luns.txt >> /tmp/netappvolumes.txt; done

This would give me a list like this:

/vol/NewYork_rvbd_d_8f3a7b69_05f7_4be8_b3a6_14a689c2b3b0_1452834001/lunC11 60.0g (64445480960) (r/w, offline)
Comment: “Cdrive”
Serial#: P3PdB/-KWreM
Share: none
Space Reservation: disabled

With that list, I can cut the volumes out with the following command:

grep -v : /tmp/netappvolumes.txt | cut -f1 -d' ' > /tmp/volumes.txt

This would give me a list like this:

/vol/NewYork_rvbd_d_8f3a7b69_05f7_4be8_b3a6_14a689c2b3b0_1452834001/lunC11

Now that I have a list of volume names from the NetApp, I can just put them all online with a loop:

for i in `cat /tmp/volumes.txt`; do echo "lun online" $i >> /tmp/online_vols.txt ; done

You can just take the /tmp/online_vols.txt file now and just paste it into your NetApp SSH session and you’ll have all of your LUNs online again.