2013年7月7日 星期日

LINUX SYSTEM INITIALIZATION

DEBIAN

1.BIOS
   |--- POST (Power On Self Test)
    `--- Load initial code (bootloader code/kernel code...etc) form DISK
2.Bootloader (Bootstrap Loader)
    `--- load system kernel image and initrd image, hands over control
`-- Linux kernel             `---- initramfs
    * initrd : image of temporary root file system loaded into memory
               do hardware detection,module loading, mnt root file sys
3. initrd (mini-Debian System (Started by bootloader))
`----- /init is executed
4. Debian-System
|----- 0. /sbin/init (init(8)) is executed
| |--- parent of all processes
| `--- create process from script /etc/inittab
|----- 1. System goes into runlevel N(none) by following /etc/inittab
|----- 2. System goes into runlevel S for initialization
`----- 3. System goes into runlevel 2-5(multi-user))(determined by /etc/inittab)
`-- rc.local is executed at the end of each multiuser runlevel




A Bootable Linux Image

Load Sequence: Boot Loader->kernel->initrd->rootfs(the fs user will be working on)

2013年7月4日 星期四

BASIC APACHE

APACHE 2 ON **DEAIBN


#       /etc/apache2/
#       |-- apache2.conf
#       |       `--  ports.conf
#       |-- mods-enabled
#       |       |-- *.load
#       |       `-- *.conf
#       |-- conf.d
#       |       `-- *
#       `-- sites-enabled
#               `-- *


**Struture of config in Debian is different from other linux dist.
** apache2.conf is the main config file
it includes other files (mods-enabled/*, conf.d/*, sites-enabled,ports.conf)

** ports.conf
ports to listen to

Directives: Timeout, Keepalive,AccessFileName ........
Configuration Section Containers:
|
|---evaluted each request : <VirtualHost> <Directory>....
| |
| |--Filesystem cotainer:<Directory> <Files>.....
| |
| `--Webspace Container: <Location>......
|
`---evaluated at startup: <IfDefine> <IfModule> ....
Directives Scoping: Limit the scope of effect of  directives(w/SectionContainer)

.htaccess: Placed in DocumentDirectory, use directives to immediately affect access
Administrator can limit directives in .htaccess by AllowOverride Directive



Virtual Hosting


How apache selects virtual to respond to request
1. Find best matching <VirtualHost> based on IP addr
2. More than 1 match--> compare ServerName and ServerAlias
3. No match found --> handled by global server config


#sites-available/000-example
<VirtualHost *:80>
             `--- Placing IP rather than DN in this directive
        ServerName www.example.org  ß FQDN of server
        ServerAdmin webmaster@localhost  ß admin mail
        DirectoryIndex index.html index.cgi index.pl index.php index.xhtml
                                                                                        `--- index files to look for
        DocumentRoot /var/www           ß root directory of site
        Alias /mydoc /var/local/lib/mydata
        <Directory />
                Options FollowSymLinks
                AllowOverride None
        </Directory>
        <Directory /var/www/>
                Options Indexes FollowSymLinks MultiViews
                                        `----Additional option for Directory
                AllowOverride None
                                        `----.htaccess (Only available under Directory)
                Order allow,deny
                allow from all
                                        |----.Search allow first, then deny, deny overwrites allow
                                        `----Rule not found --> deny
        </Directory>

        ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/  <--CGI Scripts location
        <Directory "/usr/lib/cgi-bin">
                AllowOverride None
                Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
                               |------ExecCGI à CGI script
                                      |------MultiViews HTTP1.1 content negotiation
                                       `------SymLinksIfOwnerMatch stricter symlink follow
                Order allow,deny
                Allow from all
        </Directory>

        ErrorLog ${APACHE_LOG_DIR}/error.log

        # Possible values include: debug, info, notice, warn, error, crit,
        # alert, emerg.
        LogLevel warn

        CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

<VirtualHost *:80>
    ServerName other.example.com  ßanother virtual host
    ServerAlias other.example.com *.other.example.com
                          `--another way is set ServerAlias
    DocumentRoot /www/otherdomain

</VirtualHost>


<VirtualHost _default_:80>  <--catching request not matching any vhost
    DocumentRoot /www/default

</VirtualHost>

Listen 8080   <--Listen directive should be placed here or in port.conf
Listen 9090
<VirtualHost *:8080 *:9090>  
    DocumentRoot /www/default


</VirtualHost>

UserDir Module
Mods-available/userdir.conf
<IfModule mod_userdir.c>
  UserDir public_html
  UserDir disabled root

  <Directory /home/*/public_html>
    AllowOverride FileInfo AuthConfig Limit
    Options MultiViews Indexes SymLinksIfOwnerMatch IncludesNoExec
  </Directory>
</IfModule>

CGI Dynamic Content w/ PHP,Python,Perl
$apt-get install libapache2-mod-[php,perl,python]

.htaccess

*place directives
<Files ~ "^\.(htaccess|htpasswd)$">
deny from all
</Files>
Option +Indexes –FollowSymLinks
IndexOptions +FancyIndexing
IndexIgnore *.php *.exe

*Rewirte Request URL (Requires mod_rewrite and override allowed)
.htaccess

RewriteEngine on
RewriteBase /myApp         <---- base URL for per-directory rewrites

RewriteCond %{HTTP_HOST}   !^www\.example\.com [NC]
    `----only rewrite when HOST matches(NC flag = no case)
RewriteCond %{HTTP_HOST}   !^$
RewriteCond %{SERVER_PORT} !^80$

RewriteRule ^/?(.*)         http://www.example.com:%{SERVER_PORT}/$1 [L,R,NE]
    |---- $N (0 <= N <= 9) -> back references
    |---- (flag: L -> Stop the rewriting process immediately and don't apply any more rules)
    |---- (flag: R -> Forces an external redirect)
    `----( flag: NE: noescap -> especial characters, such as & and ?, for example, will be converted to their hexcode equivalent. Using the [NE] flag prevents that from happening.)

2013年7月3日 星期三

SQL

login
mysql -hlocalhost -u[username] -p[password]

SELECT DATABASE
use [database]


CREATING USERS/ GRANTING ACCESS TO DATABASE


Create new table:

CREATE table [table name](
id int NOT NULL PRIMARY KEY,
title varchar(255) UNIQUE
.
.
.

);


**NOT NULL
**PRIMARY KEY
**UNIQUE


Insert if not entry not existent or do nothing if entry is duplicate


INSERT INTO table(c1,c2)VALUES(v1,v2) ON DUPLICATE KEY UPDATE id=LAST_INSERT_ID(id)


Search for duplicate:

SELECT address, count(id) as cnt FROM list
GROUP BY address HAVING cnt > 1


SELECT * FROM all_papers WHERE title IN(SELECT title FROM all_papers WHERE title NOT LIKE  '%unavailable%' GROUP BY (title) HAVING COUNT( title ) >1)

PHP with SQL
$USER = [username];
$PASS = [password];
try {
    $dsn = "mysql:dbname=$DBNAME;host=$HOST";
    $dbh = new PDO($dsn, $USER, $PASS);
    } 
    catch (PDOException $e) {
        print "Error!: " . $e->getMessage();
        die();
    }
    $TABLE = [tablename];
    $sql = "SELECT * FROM $TABLE";
    foreach (self::$dbh->query($sql) as $row) {
        .
        .
        .
    }

use addslashes to process string before INSERT

2013年4月21日 星期日

NFS

NFS user permissions are based on user ID (UID)
a user's access to files is determined by his/her membership of groups on the client, not on the server

Server Side

Packages

nfs-kernel-server
nfs-common
rpcbind

Server Install


apt-get install nfs-kernel-server


NFS Config

Config NFS exports so clients can mount the NFS drive

/etc/export

/home/ 192.168.100.0/24(rw,sync)

tips: Reload export config without restart NFS server

exportfs -a

restart services


/etc/init.d/rpcbind restart
/etc/init.d/nfs-kernel-server restart

Check local NFS config

showmount -e localhost

Client Side

Using autofs to mount NFS drive automatically

apt-get install autofs

File /etc/auto.master

/nfs /etc/auto.nfs

File /etc/auto.nfs

home -rw,fg,soft [IP of NFS server]:/[directory]



Mount with command manually

# mount -t nfs -o proto=tcp,port=2049 nfs-server:/ /mnt

Utililies

showmount


2013年3月26日 星期二

LDAP-NSS-PAM

LDAP
Everything is identified by dn
cn= Common Name
ou= Organizational Unit
dc= Domain Component

Schemas, objectClasses and Attributes

An objectClass is a collection of attributess:
  • An objectclass is defined within a Schema
  • An objectclass may be a part of an objectclass hierarchy, in which case it inherits all the properties of its parents.
  • An objectclass has a globally unique name or identifier

Attributes typically contain data:
  • Every attribute is defined in a schema.
  • Every attribute is included in one or moreobjectclasses.
  • To use an attribute in an entry, itsobjectclass must be included in the entry definition and its objectclass must be included in a schema. In turn, the schema must be identified to the LDAP server.
Install LDAP


apt-get install slapd ldap-utils ldapvi



Config Layout


Setup backend database

Import schemas

$ cd /etc/ldap/schema
$ ldapadd -Y EXTERNAL -H ldapi:/// -f cosine.ldif
$ ldapadd -Y EXTERNAL -H ldapi:/// -f nis.ldif
(Provides linux account attributes)
$ ldapadd -Y EXTERNAL -H ldapi:/// -f inetorgperson.ldif

backend.ldif

# Load dynamic backend modules
dn: cn=module,cn=config
objectClass: olcModuleList
cn: module
olcModulepath: /usr/lib/ldap
olcModuleload: back_hdb.la


# Database Setting
dn: olcDatabase={1}hdb,cn=config
objectClass: olcDatabaseConfig
objectClass: olcHdbConfig
olcDatabase: {1}hdb
olcSuffix: dc=[ID],dc=csie,dc=ntu,dc=edu,dc=tw
olcDbDirectory: /var/lib/ldap
olcRootDN: cn=admin,dc=[ID],dc=csie,dc=ntu,dc=edu,dc=tw
olcRootPW: {SSHA}HUpwPlcpSOwCNd8hRfiQsXFYYdyPewC2**
olcDbConfig: set_cachesize 0 2097152 0
olcDbConfig: set_lk_max_objects 1500
olcDbConfig: set_lk_max_locks 1500
olcDbConfig: set_lk_max_lockers 1500
olcDbIndex: objectClass eq
olcLastMod: TRUE
olcDbCheckpoint: 512 30
olcAccess: to attrs=userPassword,shadowLastChange
  by dn="cn=admin,dc=[ID],dc=csie,dc=ntu,dc=edu,dc=tw" write
  by anonymous auth 
  by self write
  by * none
olcAccess: to attrs=homeDirectory
  by dn="cn=admin,dc=[ID],dc=csie,dc=ntu,dc=edu,dc=tw" write
  by anonymous auth
  by self read
  by * none
olcAccess: to dn.base=""
  by * read
olcAccess: to *
  by dn="cn=admin,dc=fate,dc=csie,dc=ntu,dc=edu,dc=tw" write
  by * read


**note:olcrootPW generated by slappasswd

import backend

ldapadd -v -Y EXTERNAL -H ldapi:/// -f backend.ldif



FRONTEND: Adding users and groups (by ldapscripts or frontend.ldif file)

for /etc/passwd
objectClass: posixAccount
iduid
passworduserPassword
uiduidNumber
gidgidNumber
full_namegecos
Home DirectoryhomeDirectory
Login shellloginShell

for /etc/shadow
objectClass: shadowAccount
usernameuid
passworduserPassword
lastshadowLastChange
mayshadowMin
mustshadowMax
warnshadowWarning
expireshadowExpire
disableshadowInactive
reservedshadowFlag
for /etc/group
objectClass: posixGroup
group namecn
passworduserPassword
group idgidNumber
other accountmemberUid

option 1: Use ldapscripts to add user

install ldapscripts 
(which includes utilities too add/remove POSIX accounts)
apt-get isntall ldapscripts
modify /etc/ldapscripts/ldapscripts.conf
SERVER=localhost
BINDDN='cn=admin,dc=,dc=csie,dc=ntu,dc=edu,dc=tw'
SUFFIX='dc=,dc=csie,dc=ntu,dc=edu,dc=tw'
GSUFFIX='ou=Groups'
USUFFIX='ou=Peolple'
MSUFFIX='ou=machines'
GIDSTART=10000
UIDSTART=10000
MIDSTART=10000

PASSWORDGEN="pwgen" // $ apt-get install pwgen

RECORDPASSWORDS="yes"

PASSWORDFILE="/var/log/ldapscripts_passwd.log"

BINDPWDFILE="/etc/ldapscripts/ldapscripts.passwd"

keep the rootPW in /etc/ldapscripts/ldapscripts.passwd

sh -c "echo -n '[password]' > /etc/ldapscripts/ldapscripts.passwd"
chmod 400 /etc/ldapscripts/ldapscripts.passwd

**Careful: if authentication failure happens after ldapadduser. Check LDAP database for user password encryption

try dpkg-reconfigure libpam-ldap and set encryption method


Option 2: use frontend.ldif

file frontend.ldif

# Create top-level object in domain
dn: dc=b99902120,dc=csie,dc=ntu,dc=edu,dc=tw

objectClass: top
objectClass: dcObject
objectclass: organization
o: b99902120
dc: b99902120
description: b99902120

#people
dn: ou=people,dc=b99902120,dc=csie,dc=ntu,dc=edu,dc=tw

objectClass: organizationalUnit
ou: people

#groups
dn: ou=groups,dc=b99902120,dc=csie,dc=ntu,dc=edu,dc=tw

objectClass: organizationalUnit
ou: groups

#user setting: ta217, groups, b99902120.csie.ntu.edu.tw
dn: uid=ta217,ou=people,dc=b99902120,dc=csie,dc=ntu,dc=edu,dc=tw

objectClass: inetOrgPerson
objectClass: posixAccount
objectClass: shadowAccount
uid: ta217
displayName: ta217
uidNumber: 1001
gidNumber: 1001
userPassword: SAL2013
gecos: ta217
loginShell: /bin/bash
homeDirectory: /home/ta217
shadowExpire: -1
shadowFlag: 0
shadowWarning: 7
shadowMin: 8
shadowMax: 999999
shadowLastChange: 10877
givenName: 217
sn: ta
cn: ta217
mail: ta217@b99902120.csie.ntu.edu.tw
title: ta217

#group setting: ta217, groups, b99902120.csie.ntu.edu.tw
dn: cn=ta217,ou=groups,dc=b99902120,dc=csie,dc=ntu,dc=edu,dc=tw

objectClass: posixGroup
cn: ta217
gidNumber: 1001


import frontend file
ldapadd -v -x -D -W -f frontend.ldif


nss and pam

apt-get install libpam-ldap libnss-ldap libnss-db nslcd nscd

/etc/libnss-ldap.conf
base dc="dc=b99902120,dc=csie,dc=ntu,dc=edu,dc=tw"
/etc/nsswitch.conf 

passwd: files ldap
group: files ldap
shadow: files ldap

add users and groups
ldapadduser [user] [group]
ldapaddgroup [group]
update pam auth
pam-auth-update
auto create home directory
/etc/pam.d/common-account
session required pam_mkhomedir.so skel=/etc/skel/ umask=0022

Allow user change password
vim /etc/pam.d/common-password
password [success=1 user_unknown=ignore default=die]
pam_ldap.so use_authtok try_first_pass
change to
password [success=1 user_unknown=ignore default=die]
pam_ldap.so try_first_pass




client config

/etc/nslcd.conf
# The user and group nslcd should run as.
uid nslcd
gid nslcd

# The location at which the LDAP server(s) should be reachable.
uri ldap://192.168.50.119

# The search base that will be used for all queries.
base dc=b99902120,dc=csie,dc=ntu,dc=edu,dc=tw



/etc/nsswitch.conf
passwd: files ldap
group: files ldap
shadow: files ldap

2013年3月18日 星期一

iptables / DHCP / network interface

Interface config

/etc/network/interfaces
auto eth0
iface eth0 inet static
address [IP]
netmask [mask]
gateway [gateway(if have one)]

DHCP

Package: DHCP3-server

Select interface for DHCP server

/etc/default/dhcp3-server
INTERFACES="[interface_name]"

DHCP pools

dhcpd.conf
option domain-name "[optional domain-name]"
option domain-name-servers [DNS server IP]

e.g.
subnet 192.168.0.0 netmask 255.255.255.0{
range 192.168.0.1 192.168.0.253;
option routers 192.168.0.254;
option broadcast-address 192.168.0.255;
}

iptables / NAT

#enable ip forwarding
sysctl -w net.ipv4.ip_forward=1
#flush tables
iptables -F -t nat
iptables -F INPUT
iptables -F OUTPUT
iptables -F FORWARD
iptables -P INPUT ACCEPT
iptables -P OUTPUT ACCEPT
iptables -P FORWARD DROP
#NAT
iptables -t nat -A POSTROUTING -s 192.168.0.0/24 -o eth0 -j MASQUERADE
#firewall
iptables -A FORWARD -p icmp -j ACCEPT
iptables -A FORWARD -p udp --sport 53 -j ACCEPT
iptables -A FORWARD -p udp --dport 53 -j ACCEPT
iptables -A FORWARD -p tcp --sport 80 -j ACCEPT
iptables -A FORWARD -p tcp --dport 80 -j ACCEPT
iptables -A FORWARD -p tcp --sport 20:21 -j ACCEPT
iptables -A FORWARD -p tcp --dport 20:21 -j ACCEPT
iptables -A FORWARD -p tcp --sport 23 -j ACCEPT
iptables -A FORWARD -p tcp --dport 23 -j ACCEPT
iptables -A FORWARD -p tcp --sport 110 -j ACCEPT
iptables -A FORWARD -p tcp --dport 110 -j ACCEPT
iptables -A FORWARD -p tcp --sport 25 -j ACCEPT
iptables -A FORWARD -p tcp --dport 25 -j ACCEPT
iptables -A FORWARD -p tcp --sport 995 -j ACCEPT
iptables -A FORWARD -p tcp --dport 995 -j ACCEPT

2013年3月1日 星期五

warning: setlocale LC_ALL

What is LC_ALL:

LC_ALL is the environment variable that overrides all the other localisation settings 
The value 'LC_ALL=C' is essentially an English-only environment that specifies the ANSI C locale.

You'll typically set $LANG to your preference. The individual LC_xxx variables override a certain aspect. LC_ALL overrides them all. The locale command, when called without argument gives a summary of the current settings.

LC_* var priority

LC_ALL > LC_* > LANG

locale def file:

/usr/share/i18n/locales
vi /etc/default/locale

LANG & LANGUAGE

LANG: specifies the default locale for all unset locale variables
LANGUAGE: most programs use this for the language of its interface

warning: setlocale LC_ALL solution

export LC_ALL="en_US.UTF-8" export LANGUAGE="en_US.UTF-8"