Author Archives: saisai

Pip install from git hub

pip install git+https://github.com/frankban/django-endless-pagination.git

http://stackoverflow.com/questions/20101834/pip-install-from-github-repo-branch

$ pip install git+git://github.com/myuser/foo.git@v123
or
$ pip install git+git://github.com/myuser/foo.git@newbranch

http://stackoverflow.com/questions/8247605/configuring-so-that-pip-install-can-work-from-github

https://pip.pypa.io/en/latest/reference/pip_install.html#vcs-support

Solving Bind DNS un-resolved when nslookup domain without WWW

Solving Bind DNS un-resolved when nslookup domain without WWW

This is common mistake when we try to nslookup our domain that working with “WWW” but not without “WWW”. This is some bind9 configuration :

;
; BIND data file for local loopback interface
;
$TTL 3600
@ IN SOA obroll.com. admin.obroll.com. (
30 ; Serial
604800 ; Refresh
86400 ; Retry
2419200 ; Expire
604800 ) ; Negative Cache TTL
;
@ IN NS ns1.obroll.com.
ns1 IN A 15.185.178.20
ns2 IN A 15.185.178.20
www IN A 15.185.178.20

You will see there nothing wrong here until you realize you missing something.
Yes, I’m missing “@ IN “. Then, it should be :

;
; BIND data file for local loopback interface
;
$TTL 3600
@ IN SOA obroll.com. admin.obroll.com. (
30 ; Serial
604800 ; Refresh
86400 ; Retry
2419200 ; Expire
604800 ) ; Negative Cache TTL
;
@ IN NS ns1.obroll.com.
@ IN A 15.185.178.20
ns1 IN A 15.185.178.20
ns2 IN A 15.185.178.20
www IN A 15.185.178.20

credit : http://obroll.com/solving-bind-dns-un-resolved-when-nslookup-domain-without-www/

What does > /dev/null 2>&1 mean?

What does > /dev/null 2>&1 mean?

I remember being confused for a very long time about the trailing garbage in commands I saw in Unix systems, especially while watching compilers do their work. Nobody I asked could tell me what the funny greater-thans, ampersands and numbers after the commands meant, and search engines never turned up anything but examples of it being used without explanation. In this article I’ll explain those weird commands.

Here’s an example command:

wibble > /dev/null 2>&1

Output redirection

The greater-thans (>) in commands like these redirect the program’s output somewhere. In this case, something is being redirected into /dev/null, and something is being redirected into &1.
Standard in, out, and error

There are three standard sources of input and output for a program. Standard input usually comes from the keyboard if it’s an interactive program, or from another program if it’s processing the other program’s output. The program usually prints to standard output, and sometimes prints to standard error. These three file descriptors (you can think of them as “data pipes”) are often called STDIN, STDOUT, and STDERR.

Sometimes they’re not named, they’re numbered! The built-in numberings for them are 0, 1, and 2, in that order. By default, if you don’t name or number one explicitly, you’re talking about STDOUT.

Given that context, you can see the command above is redirecting standard output into /dev/null, which is a place you can dump anything you don’t want (often called the bit-bucket), then redirecting standard error into standard output (you have to put an & in front of the destination when you do this).

The short explanation, therefore, is “all output from this command should be shoved into a black hole.” That’s one good way to make a program be really quiet!

How to Upgrade Python in CentOS

How to Upgrade Python in CentOS

This guide will teach you how to upgrade from Python 2.4 to Python 2.7.

First you need to check your current Python Version:

python -V

If it returns 2.4 then execute the following steps:

wget http://python.org/ftp/python/2.7.3/Python-2.7.3.tgz
tar -zxvf Python-2.7.3.tgz
cd Python-2.7.3
yum install gcc
./configure –enable-shared
make
make altinstall

The above method will install Python 2.7 without overwriting the original 2.4 version of Python in CentOS (NOTE: Overwriting 2.4 completely with 2.7 is bad!)

Basically the “make altinstall” will install Python 2.7 to “/usr/local/bin/python2.7″

To check execute: “python2.7 -V” and it will show the 2.7 version while executing “python -V” will result to the 2.4 version.

So how are we going to configure CentOS to use Python 2.7 as default instead of Python 2.4?

Easy. Just follow the following:

1. Add Python 2.7 lib to system lib:

ln -s /opt/python2.7/lib/libpython2.7.so /usr/lib
ln -s /opt/python2.7/lib/libpython2.7.so.1.0 /usr/lib

2. Create important links and cache

/sbin/ldconfig -v

3. Create a symbolic link:

ln -s /opt/python2.7/bin/python /usr/local/python

4. Check version by executing “python -V”. If you still see old version proceed to step 5.

5. Modify /etc/profile (as root)

vi /etc/profile

Look for the following entry “export PATH USER LOGNAME MAIL HOSTNAME HISTSIZE INPUTRC” and put the following setting “pathmunge /opt/python2.7/bin/” above it.

It should look like this:

pathmunge /opt/python2.7/bin/
export PATH USER LOGNAME MAIL HOSTNAME HISTSIZE INPUTRC

Then re-login system (very important), try “python -V” again.

6. (Very important last step) Make sure yum is just ok by executing “yum search php”. If it doesn’t return any errors then you are all good.

Credit:http://www.nerdsmind.com/how-to-upgrade-python-in-centos/

The Digital Text

In this post we will example the digital text and how and why it is encoded.
Denarius

Wars are often an unexpected event, and a lot of the major currency developments in the 19th and 20th centuries were due to wars. In the ancient world it was no different. The requirement to quickly gather resources required an efficient form of money.

During the Second Punic War, in 211 BC, Rome brought out the Denarius, which means ‘containing ten’ – because one silver Denarius was worth ten bronze (later copper) Asses.

During the Third Punic war, in 140 BC, Rome decided to go hexadecimal, where one silver-coloured Denarius became worth 16 copper Asses.

The silver-coloured Denarius was considered a day’s wages for a soldier. The gold-coloured Solidus varied in value but eventually stabilised by the 8th century as 12 denarii.

The Romans carried spread currency around and in Britain, the denarius became the penny but was still written as d until 1971 e.g. 5d for 5 pence.

12d made a shilling, which is the Anglo-Saxon term for the Solidus. The shilling was in the 16th century pegged to the value of a cow in Kent market.

Twenty shilling made a pound which was named after the unit of mass, a pound in cash was originally worth the value of a pound weight of silver (which is now about £300).

The pound of a unit of mass is itself Roman of course, from libra, which is why pound is shortened to lb. The pound £ sign was originally an L. 1 lb in mass is 16 ounces.

Part the deal when Britain applied to join the European Economic Community in the 1960s and 1970s, was that we got rid of all these crazy measurements and adopted metric, also known as scientific measurements, which we did eventually, to a certain extent. For example, milk, beer and cider are officially sold in units of 568 mL!

So until recently, the idea of non-base 10 measurements was completely normal.
Binary

George Boole was a theologian who was also one of the greatest mathematicians of the 19th Century.Boole understood mathematics and religion as intertwined. George Boole believed that studying mathematics would help reveal a new understanding of God.
../../../_images/george_boole.jpg

More on George Boole: http://zeth.net/archive/2007/07/19/what-is-truth-part-3-all-you-need-is-one-and-zero/

The core idea that all knowledge and thought could be reduced to two factors nothing (0) and God (1), had long been discussed, for example by the the Jesuit Gottfried Leibniz writing in the 17th Century. However, Boole had the mathematical knowledge to take the idea and build a complete system of logic around it.
../../../_images/yin_and_yang.png

Everything is either True (God – 1) or False (nothing – 0):

1 or 0 == 1
0 or 1 == 1
1 or 1 == 0
0 or 0 == 0
1 and 1 == 1
0 and 0 == 0
1 and 0 == 0
0 and 1 == 0
not 0 == 1
not 1 == 0

Everything that is not God is nothingness, everything that is something is God. God fills the nothingness but the nothingness cannot conquer God.

Any number can be represented by any sequence of bits. A bit is 0 or a 1.
Binary Decimal
0 0
1 1
10 2
11 3
100 4
101 5
110 6
111 7
1000 8
1001 9
1010 10
1011 11
1100 12
1101 13
1110 14
1111 15
10000 16

Traditionally, eight bits was called a byte (more correctly it is an octet). Four bits is a nibble.

A computer processor has lots of microscopic transistors. The CPU in my laptop (the Intel Ivy Bridge) has 1.4 billion of them. Each transistor is like a switch with an on and off state.
Hexadecimal

Binary is very low level. The first level of abstraction over binary is called hexadecimal.

In previous lecture, we looked at how and when and where computing was developed. These early computer developers choose the most efficient representation. As we mentioned earlier, until recently non-base 10 measurements were completely normal.

Hexadecimal (‘hex’ for short) is counting in base 16, here is the table from above with hex as well:
Binary Hex Decimal
0 0 0
1 1 1
10 2 2
11 3 3
100 4 4
101 5 5
110 6 6
111 7 7
1000 8 8
1001 9 9
1010 a 10
1011 b 11
1100 c 12
1101 d 13
1110 e 14
1111 f 15
10000 10 16

Now it is easy to convert any binary number to hex. You just split it up into nibbles from the right.

So this number:

11111011110

Split up is:

0111 1101 1110
7 d e

So in hex it is 7de.

What number is it in decimal? Well that is more complicated. Going from binary to decimal requires you to split the binary number up into parts:

10000000000 1024
1000000000 512
100000000 256
10000000 128
1000000 64
10000 16
1000 8
100 4
10 2

1024 + 512 + 256 + 128 + 64 + 16 + 8 + 4 + 2 = ?

So data is electrical impulses in a transistor, which represent 1 and 0, which are then hexadecimal numbers.

Now we have numbers, we can now encode characters. Each character is given a hex number.

So 41 in hex (which is 65 in decimal) is “latin capital letter A”.

There are different encodings (mapping between numbers and characters) but the only one that really matters in 2014 is called UTF-8 commonly called Unicode (although there are other forms of Unicode which did not win).

UTF-8 has room for 1,112,064 different characters and symbols which aim to represent all of the world’s languages.

The first 128 characters are carried over from an older standard called ASCII. The first 32 of these are historic control characters for controlling printers and teletype devices (remember those from a previous lecture?).

20 in hex (so 32 in decimal) is the empty space, then we get punctuation, then we get the numbers and so more punctuation etc then the letters in upper case then some more symbols then the letters in lower case etc.

This gets us to 7E (126) which is ~, and we have all of the English keyboard covered. The next 129 characters are Western European languages (German etc) and then it carries on after that through all the world’s letters and symbols.

Including some really fun stuff added to give compatibility with Japanese mobile phones:

http://www.fileformat.info/info/unicode/char/1f4a9/index.htm http://www.fileformat.info/info/unicode/char/1f302/index.htm http://www.fileformat.info/info/unicode/block/miscellaneous_symbols_and_pictographs/images.htm http://en.wikipedia.org/wiki/Emoji

So a digital text is a series of these hexadecimal numbers representing characters and symbols including spaces (20 in hex/32 in decimal) and control codes such as line breaks (0A in hex, 10 in decimal) and so on.

Here is a nice chart version of the first 127 (ASCII) characters: http://web.cs.mun.ca/~michael/c/ascii-table.html

So you can decode these characters (with some spaces added to make it simpler):

41 6e 64 20 74 68 65 72 65 66 6f 72 65 20 6e 65 76 65 72 20 73 65 6e 64 20 74 6f 20 6b 6e 6f 77 20 66 6f 72 20 77 68 6f 6d 20 74 68 65 20 62 65 6c 6c 20 74 6f 6c 6c 73 3b 20 49 74 20 74 6f 6c 6c 73 20 66 6f 72 20 74 68 65 65 2e

To make it clear that something is a hex value, it is often prefixed with 0x or x or U+.

This is as good as it far as it goes. But to make practical use of the data, just loads of text doesn’t help that much.

If we want to make a digital representation of a humanities artefact like a manuscript, we need to use a file format. Otherwise the digital text is of limited use for other scholars and software.

Credit to

http://commandline.org.uk/

Installing Django with Apache and mod_wsgi on Ubuntu 10.04

Installing Django with Apache and mod_wsgi on Ubuntu 10.04
By Kevan Stannard | Published: December 11, 2010

Step by step instructions for installing Django with Apache and mod_wsgi on Ubuntu 10.04.

PART 1 – Prepare the server

Update the server

> sudo apt-get update
> sudo apt-get upgrade

Install Apache and mod_wsgi

> sudo apt-get install apache2 libapache2-mod-wsgi

Install setup tools and pip

> sudo apt-get install python-setuptools
> sudo apt-get install python-pip

Install Django

> sudo pip install django

Create a folder for storing our sites

I’ll be placing our sites in the /srv/www directory. The /srv directory should already exist so we just need to create the /www directory

> sudo mkdir /srv/www

PART 2 – Add host entries for testing

We will set up two domains for testing the configuration
- one for testing that WSGI is working, and
- one for testing that Django is working.

My test virtual machine’s IP address is 172.16.52.130 so I’ll set up the following in my local hosts file (not on the server)

> sudo nano /etc/hosts

And add the following

172.16.52.130 djangoserver
172.16.52.130 wsgi.djangoserver
172.16.52.130 hello.djangoserver

PART 3 – Test WSGI is working

Create our wsgi test site content

> sudo mkdir /srv/www/wsgi
> sudo nano /srv/www/wsgi/app.wsgi

And add the content

def application(environ, start_response):
status = ’200 OK’
output = ‘Hello World!’

response_headers = [('Content-type', 'text/plain'),
('Content-Length', str(len(output)))]
start_response(status, response_headers)

return [output]

Create a new apache site

> sudo nano /etc/apache2/sites-available/wsgi

And add the content

ServerName wsgi.djangoserver
DocumentRoot /srv/www/wsgi


Order allow,deny
Allow from all

WSGIScriptAlias / /srv/www/wsgi/app.wsgi

And activate the site

> sudo a2ensite wsgi
> sudo /etc/init.d/apache2 reload

Then open your web browser and browse to

http://wsgi.djangoserver

You should see a ‘Hello World!’ message
PART 4 – Test Django is working

Create a new Django site

> cd /srv/www
> sudo django-admin.py startproject hello

Create a wsgi file for the site

> sudo mkdir /srv/www/hello/apache
> sudo nano /srv/www/hello/apache/django.wsgi

And add the content

import os
import sys

path = ‘/srv/www’
if path not in sys.path:
sys.path.insert(0, ‘/srv/www’)

os.environ['DJANGO_SETTINGS_MODULE'] = ‘hello.settings’

import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

Create a new apache site

> sudo nano /etc/apache2/sites-available/hello

And add the content

ServerName hello.djangoserver
DocumentRoot /srv/www/hello


Order allow,deny
Allow from all

WSGIDaemonProcess hello.djangoserver processes=2 threads=15 display-name=%{GROUP}
WSGIProcessGroup hello.djangoserver

WSGIScriptAlias / /srv/www/hello/apache/django.wsgi

And activate the site

> sudo a2ensite hello
> sudo /etc/init.d/apache2 reload

Then open your web browser and browse to

http://hello.djangoserver

You should see the Django default installation message.
Notes

NOTE 1 – Running in daemon mode

Our test Django site is configured to run in daemon mode – because of these two lines:

WSGIDaemonProcess hello.djangoserver processes=2 threads=15 display-name=%{GROUP}
WSGIProcessGroup hello.djangoserver

So if we modify the code then rather than restarting apache we can just touch the wsgi file and the changes will be picked up:

> sudo touch /srv/www/hello/apache/django.wsgi

NOTE 2 – Specify the application module name

It appears to be a good idea to specify the application module when specifying the DJANGO_SETTINGS_MODULE. So rather than writing this:

os.environ['DJANGO_SETTINGS_MODULE'] = ‘settings’

We should write this:

os.environ['DJANGO_SETTINGS_MODULE'] = ‘hello.settings’

Useful Commands

Error log file

If you get errors, check the apache log file

> tail /var/log/apache2/error.log

Test using development mode

If your app does not seem to be working using wsgi, then check if it is working via the development server.

> cd /srv/www/hello
> python manage.py runserver 0:8080

The in your web browser go to

http://hello.djangoserver:8080

Reference:

http://blog.stannard.net.au/2010/12/11/installing-django-with-apache-and-mod_wsgi-on-ubuntu-10-04/

http://thecodeship.com/deployment/deploy-django-apache-virtualenv-and-mod_wsgi/

Relaying Postfix mails via smtp.gmail.com

sudo apt-get install postfix mailutils libsasl2-2 ca-certificates libsasl2-modules

vim /etc/postfix/main.cf
relayhost = [smtp.gmail.com]:587
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous
smtp_tls_CAfile = /etc/postfix/cacert.pem
smtp_use_tls = yes

vim /etc/postfix/sasl_passwd
[smtp.gmail.com]:587 USERNAME@gmail.com:PASSWORD

sudo chmod 400 /etc/postfix/sasl_passwd
sudo postmap /etc/postfix/sasl_passwd

/etc/ssl/certs/Thawte_Premium_Server_CA.pem | sudo tee -a /etc/postfix/cacert.pem

sudo /etc/init.d/postfix reload

echo “Test mail from postfix” | mail -s “Test Postfix” you@example.com

Reference:

https://rtcamp.com/tutorials/linux/ubuntu-postfix-gmail-smtp/

mysql2sqlite.sh

#!/bin/sh

# Converts a mysqldump file into a Sqlite 3 compatible file. It also extracts the MySQL `KEY xxxxx` from the
# CREATE block and create them in separate commands _after_ all the INSERTs.

# Awk is choosen because it’s fast and portable. You can use gawk, original awk or even the lightning fast mawk.
# The mysqldump file is traversed only once.

# Usage: $ ./mysql2sqlite mysqldump-opts db-name | sqlite3 database.sqlite
# Example: $ ./mysql2sqlite –no-data -u root -pMySecretPassWord myDbase | sqlite3 database.sqlite

# Thanks to and @artemyk and @gkuenning for their nice tweaks.

mysqldump –compatible=ansi –skip-extended-insert –compact “$@” | \

awk ‘

BEGIN {
FS=”,$”
print “PRAGMA synchronous = OFF;”
print “PRAGMA journal_mode = MEMORY;”
print “BEGIN TRANSACTION;”
}

# CREATE TRIGGER statements have funny commenting. Remember we are in trigger.
/^\/\*.*CREATE.*TRIGGER/ {
gsub( /^.*TRIGGER/, “CREATE TRIGGER” )
print
inTrigger = 1
next
}

# The end of CREATE TRIGGER has a stray comment terminator
/END \*\/;;/ { gsub( /\*\//, “” ); print; inTrigger = 0; next }

# The rest of triggers just get passed through
inTrigger != 0 { print; next }

# Skip other comments
/^\/\*/ { next }

# Print all `INSERT` lines. The single quotes are protected by another single quote.
/INSERT/ {
gsub( /\\\047/, “\047\047″ )
gsub(/\\n/, “\n”)
gsub(/\\r/, “\r”)
gsub(/\\”/, “\””)
gsub(/\\\\/, “\\”)
gsub(/\\\032/, “\032″)
print
next
}

# Print the `CREATE` line as is and capture the table name.
/^CREATE/ {
print
if ( match( $0, /\”[^\"]+/ ) ) tableName = substr( $0, RSTART+1, RLENGTH-1 )
}

# Replace `FULLTEXT KEY` or any other `XXXXX KEY` except PRIMARY by `KEY`
/^ [^"]+KEY/ && !/^ PRIMARY KEY/ { gsub( /.+KEY/, ” KEY” ) }

# Get rid of field lengths in KEY lines
/ KEY/ { gsub(/\([0-9]+\)/, “”) }

# Print all fields definition lines except the `KEY` lines.
/^ / && !/^( KEY|\);)/ {
gsub( /AUTO_INCREMENT|auto_increment/, “” )
gsub( /(CHARACTER SET|character set) [^ ]+ /, “” )
gsub( /DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP|default current_timestamp on update current_timestamp/, “” )
gsub( /(COLLATE|collate) [^ ]+ /, “” )
gsub(/(ENUM|enum)[^)]+\)/, “text “)
gsub(/(SET|set)\([^)]+\)/, “text “)
gsub(/UNSIGNED|unsigned/, “”)
if (prev) print prev “,”
prev = $1
}

# `KEY` lines are extracted from the `CREATE` block and stored in array for later print
# in a separate `CREATE KEY` command. The index name is prefixed by the table name to
# avoid a sqlite error for duplicate index name.
/^( KEY|\);)/ {
if (prev) print prev
prev=””
if ($0 == “);”){
print
} else {
if ( match( $0, /\”[^"]+/ ) ) indexName = substr( $0, RSTART+1, RLENGTH-1 )
if ( match( $0, /\([^()]+/ ) ) indexKey = substr( $0, RSTART+1, RLENGTH-1 )
key[tableName]=key[tableName] “CREATE INDEX \”” tableName “_” indexName “\” ON \”” tableName “\” (” indexKey “);\n”
}
}

# Print all `KEY` creation lines.
END {
for (table in key) printf key[table]
print “END TRANSACTION;”
}

exit 0

reference:

https://gist.github.com/esperlu/943776#file-mysql2sqlite-sh

$ ./mysql2sqlite.sh -u MyUserName -pMySecretPassWord myDbase | sqlite3 database.sqlite