0) Backup your database.
You should probably be doing this already.  Now’s a good time to make sure that your backups ran.

1) Create the script.
You’ll need the correct permissions to query the database. Here’s the command.  Be sure to change <DATABASE_NAME> as it fits.

# mysql -p -e "show tables in <DATABASE_NAME>;" | \
tail --lines=+2 | \
xargs -i echo "ALTER TABLE {} ENGINE=INNODB;" > alter_table.sql

2) Run the script.

# mysql --database=<DATABASE_NAME> -p < alter_table.sql

3) Verify it by running this command in mysql:

mysql> show table status;

SOURCE

/usr/local/bin/mysql.backup.sh

#!/bin/bash
NOW=$(date +"%m-%d-%Y")
OLD=$(date +"%m-%d-%Y" --date="3 days ago")
PROJECT="project_name"
LOCATION="/home/backup"
FILE="$PROJECT.$NOW.sql"
FILE2="$FILE.gz"
FILEOLD="$PROJECT.$OLD.sql.gz"
EMAIL="youremail@domain.com"
$SQLUSER="username"
$SQLPASS="password"
$SQLNAME="database_name"

cd $LOCATION ; \
rm -f $FILEOLD ; \
mysqldump -u $SQLUSER --password=$SQLPASS $SQLNAME > \
$LOCATION/$FILE ; \
gzip $LOCATION/$FILE ; \
echo "Backup location is in $LOCATION/$FILE2" | \
mail -s "[$PROJECT] MySQL Backup" $EMAIL

Then you can put it on your cron (background process)

Below cron will execute the script on Saturday at 12AM:

0 0 * * 6 /usr/local/bin/mysql.backup.sh

Good luck!

When invoked without arguments, the date command displays the current date and time. Depending on the options specified, date will set the date and time or print it in a user defined way. I’ve seen many people writing a perl script for calculating yesterday or tomorrow. Computer loves numbers but we love relative terms like 2 days ago. Luckily GNU date command is designed to handle relative date calculation.

Why use relative date formats?

• Ease of use
• To write your own scripts
• Automate task using cron (example run a job on last day of the month or Nth day of the month or 3rd Friday and so on)

First, print today’s date:
$ date
Sun Jun 17 12:17:24 CDT 2007

Now display Yesterday’s date:
$ date --date="1 days ago"
OR try:
$ date --date="yesterday"
Sat Jun 16 12:17:20 CDT 2007

Now display Tomorrow’s date:
$ date --date="-1 days ago"
Or better try:
$ date --date="next day"
Sat Jun 16 12:17:20 CDT 2007

Getting date in the future

To get tomorrow and day after tomorrow (tomorrow+N) use day word to get date in the future.

Getting date in the past

To get yesterday and earlier day in the past use string day ago:

Moving by whole years or months

You can add year and months keywords to get more accurate date:
$ date --date='2 year ago' # past
$ date --date='3 years' # go into future
$ date --date='2 days' # future
$ date --date='1 month ago' # past
$ date --date='2 months' # future

Moving date using more precise units

• You can use fortnight for 14 day
• Week for 7 days
• hour for 60 minutes
• minute for 60 seconds
• second for one second
• You can also use this / now / today keywords to stress the meaning

To print the date of this Friday:
$ date --date='this Friday'
To print the date of the day six months and 15 day
$ date --date='6 months 15 day'
To print the date of the day two months and 5 days ago:
$ date --date='2 months 5 day ago'

You can also use relative format to setup date and time. For example to set the system clock forward by 30 minutes, enter:
# date --set='+30 minutes'

To display date in epoch time:
$ date --date='1970-01-01 00:00:01 UTC +5 hours' +%s

SOURCE

Q. How do I format date to display on screen on for my scripts as per my requirements?

A. You need to use standard date command to format date or time for output or to use in a shell script.

Syntax to specify format
date +FORMAT

Task: Display date in mm-dd-yy format

Type the command as follows:
$ date +"%m-%d-%y"
Output:

02-27-07

Turn on 4 digit year display:
$ date +"%m-%d-%Y"
Just display date as mm/dd/yy format:
$ date +"%D"

Task: Display time only

Type the command as follows:
$ date +"%T"
Output:

19:55:04

Display locale’s 12-hour clock time
$ date +"%r"
Output:

07:56:05 PM

Display time in HH:MM format:
$ date +"%H-%M"

How do I save time/date format to a variable?

Simply type command as follows at a shell prompt:
$ NOW=$(date +"%m-%d-%Y")
To display a variable use echo / printf command:
$ echo $NOW
Sample shell script:

#!/bin/bash
NOW=$(date +"%m-%d-%Y")
FILE="backup.$NOW.tar.gz"
# rest of script

Complete list of FORMAT control characters supported by date command

FORMAT controls the output.It can be the combination of any one of the following:

%%

a literal %

%a

locale’s abbreviated weekday name (e.g., Sun)

%A

locale’s full weekday name (e.g., Sunday)

%b

locale’s abbreviated month name (e.g., Jan)

%B

locale’s full month name (e.g., January)

%c

locale’s date and time (e.g., Thu Mar 3 23:05:25 2005)

%C

century; like %Y, except omit last two digits (e.g., 21)

%d

day of month (e.g, 01)

%D

date; same as %m/%d/%y

%e

day of month, space padded; same as %_d

%F

full date; same as %Y-%m-%d

%g

last two digits of year of ISO week number (see %G)

%G

year of ISO week number (see %V); normally useful only with %V

%h

same as %b

%H

hour (00..23)

%I

hour (01..12)

%j

day of year (001..366)

%k

hour ( 0..23)

%l

hour ( 1..12)

%m

month (01..12)

%M

minute (00..59)

%n

a newline

%N

nanoseconds (000000000..999999999)

%p

locale’s equivalent of either AM or PM; blank if not known

%P

like %p, but lower case

%r

locale’s 12-hour clock time (e.g., 11:11:04 PM)

%R

24-hour hour and minute; same as %H:%M

%s

seconds since 1970-01-01 00:00:00 UTC

%S

second (00..60)

%t

a tab

%T

time; same as %H:%M:%S

%u

day of week (1..7); 1 is Monday

%U

week number of year, with Sunday as first day of week (00..53)

%V

ISO week number, with Monday as first day of week (01..53)

%w

day of week (0..6); 0 is Sunday

%W

week number of year, with Monday as first day of week (00..53)

%x

locale’s date representation (e.g., 12/31/99)

%X

locale’s time representation (e.g., 23:13:48)

%y

last two digits of year (00..99)

%Y

year

%z

+hhmm numeric timezone (e.g., -0400)

%:z

+hh:mm numeric timezone (e.g., -04:00)

%::z

+hh:mm:ss numeric time zone (e.g., -04:00:00)

%:::z

numeric time zone with : to necessary precision (e.g., -04, +05:30)

%Z

alphabetic time zone abbreviation (e.g., EDT)

SOURCE

See also:

• Shell Scripting: Creating report/log file names with date in filename

This is a quick walk through on how to set up domain keys on Centos 5 using sendmail. It should also be very similar for Redhat or Fedora.

First, Squid server installed (use up2date squid) and configured by adding following directives to file:
# vi /etc/squid/squid.conf

Modify or add following squid directives:
httpd_accel_host virtual
httpd_accel_port 80
httpd_accel_with_proxy on
httpd_accel_uses_host_header on
acl lan src 192.168.1.1 192.168.2.0/24
http_access allow localhost
http_access allow lan

Where,

• httpd_accel_host virtual: Squid as an httpd accelerator
• httpd_accel_port 80: 80 is port you want to act as a proxy
• httpd_accel_with_proxy on: Squid act as both a local httpd accelerator and as a proxy.
• httpd_accel_uses_host_header on: Header is turned on which is the hostname from the URL.
• acl lan src 192.168.1.1 192.168.2.0/24: Access control list, only allow LAN computers to use squid
• http_access allow localhost: Squid access to LAN and localhost ACL only
• http_access allow lan: — same as above –

Here is the complete listing of squid.conf for your reference (grep will remove all comments and sed will remove all empty lines, thanks to David Klein for quick hint ):
# grep -v "^#" /etc/squid/squid.conf | sed -e '/^$/d'

OR, try out sed (thanks to kotnik for small sed trick)
# cat /etc/squid/squid.conf | sed '/ *#/d; /^ *$/d'

Output:
hierarchy_stoplist cgi-bin ?
acl QUERY urlpath_regex cgi-bin \?
no_cache deny QUERY
hosts_file /etc/hosts
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern . 0 20% 4320
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl purge method PURGE
acl CONNECT method CONNECT
cache_mem 1024 MB
http_access allow manager localhost
http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
acl lan src 192.168.1.1 192.168.2.0/24
http_access allow localhost
http_access allow lan
http_access deny all
http_reply_access allow all
icp_access allow all
visible_hostname myclient.hostname.com
httpd_accel_host virtual
httpd_accel_port 80
httpd_accel_with_proxy on
httpd_accel_uses_host_header on
coredump_dir /var/spool/squid

Iptables configuration

Next, I had added following rules to forward all http requests (coming to port 80) to the Squid server port 3128 :
iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 80 -j DNAT --to 192.168.1.1:3128
iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 3128

Here is complete shell script. Script first configure Linux system as router and forwards all http request to port 3128 (Download the fw.proxy shell script):
#!/bin/sh
# squid server IP
SQUID_SERVER="192.168.1.1"
# Interface connected to Internet
INTERNET="eth0"
# Interface connected to LAN
LAN_IN="eth1"
# Squid port
SQUID_PORT="3128"
# DO NOT MODIFY BELOW
# Clean old firewall
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
# Load IPTABLES modules for NAT and IP conntrack support
modprobe ip_conntrack
modprobe ip_conntrack_ftp
# For win xp ftp client
#modprobe ip_nat_ftp
echo 1 > /proc/sys/net/ipv4/ip_forward
# Setting default filter policy
iptables -P INPUT DROP
iptables -P OUTPUT ACCEPT
# Unlimited access to loop back
iptables -A INPUT -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT
# Allow UDP, DNS and Passive FTP
iptables -A INPUT -i $INTERNET -m state --state ESTABLISHED,RELATED -j ACCEPT
# set this system as a router for Rest of LAN
iptables --table nat --append POSTROUTING --out-interface $INTERNET -j MASQUERADE
iptables --append FORWARD --in-interface $LAN_IN -j ACCEPT
# unlimited access to LAN
iptables -A INPUT -i $LAN_IN -j ACCEPT
iptables -A OUTPUT -o $LAN_IN -j ACCEPT
# DNAT port 80 request comming from LAN systems to squid 3128 ($SQUID_PORT) aka transparent proxy
iptables -t nat -A PREROUTING -i $LAN_IN -p tcp --dport 80 -j DNAT --to $SQUID_SERVER:$SQUID_PORT
# if it is same system
iptables -t nat -A PREROUTING -i $INTERNET -p tcp --dport 80 -j REDIRECT --to-port $SQUID_PORT
# DROP everything and Log it
iptables -A INPUT -j LOG
iptables -A INPUT -j DROP

Save shell script. Execute script so that system will act as a router and forward the ports:
# chmod +x /etc/fw.proxy
# /etc/fw.proxy
# service iptables save
# chkconfig iptables on

Start or Restart the squid:
# /etc/init.d/squid restart
# chkconfig squid on

Desktop / Client computer configuration

Point all desktop clients to your eth1 IP address (192.168.2.1) as Router/Gateway (use DHCP to distribute this information). You do not have to setup up individual browsers to work with proxies.

How do I test my squid proxy is working correctly?

See access log file /var/log/squid/access.log:
# tail -f /var/log/squid/access.log

Above command will monitor all incoming request and log them to /var/log/squid/access_log file. Now if somebody accessing a website through browser, squid will log information.

Problems and solutions

(a) Windows XP FTP Client

All Desktop client FTP session request ended with an error:
Illegal PORT command.

I had loaded the ip_nat_ftp kernel module. Just type the following command press Enter and voila!
# modprobe ip_nat_ftp

Please note that modprobe command is already added to a shell script (above).

(b) Port 443 redirection

I had block out all connection request from our router settings except for our proxy (192.168.1.1) server. So all ports including 443 (https/ssl) request denied. You cannot redirect port 443, from debian mailing list, “Long answer: SSL is specifically designed to prevent “man in the middle” attacks, and setting up squid in such a way would be the same as such a “man in the middle” attack. You might be able to successfully achive this, but not without breaking the encryption and certification that is the point behind SSL“.

Therefore, I had quickly reopen port 443 (router firewall) for all my LAN computers and problem was solved.

(c) Squid Proxy authentication in a transparent mode

You cannot use Squid authentication with a transparently intercepting proxy.

Further reading:

• How do I use Iptables connection tracking feature?
• How do I build a Simple Linux Firewall for DSL/Dial-up connection?
• Update: Forum topic discussion: Setting up a transparent proxy with Squid peering to ISP squid server
• Squid, a user’s guide
• Squid FAQ
• Transparent Proxy with Linux and Squid mini-HOWTO

Source

Most of them involve changing the daemon’s configuration in /etc/sendmail.mc and rebuilding sendmail.cf

Specific things that can affect performance:

dnl # Sendmail, Chap 24.9.13, Page 955
dnl # Disable re-write of queue control file (will result in duplicates
dnl #   if the daemon is interrupted during a delivery)
define(`confCHECKPOINTINTERVAL’,`0′)dnl

dnl # Sendmail, Chap 24.9.21, Page 960
dnl # Disable throttling the acceptance of new connections
define(`confCONNECTION_RATE_THROTTLE’,`0′)dnl

dnl # Sendmail, Chap 24.9.25, Page 967
dnl # Specify the maximum size, in bytes, of buffered df* files (default is
dnl #   4096 bytes; 0 turns this off and is not recommended)
define(`confDF_BUFFER_SIZE’,`16384′)dnl

dnl # Sendmail, Chap 24.9.60, Page 1011
dnl # Disable limit on the daemon spawning new children
define(`confMAX_DAEMON_CHILDREN’,`0′)dnl

dnl # Sendmail, Chap 24.9.66, Page 1016
dnl # Disbale limit on the number of messages that may be processed
dnl #  during any one queue run
define(`confMAX_QUEUE_RUN_SIZE’,`0′)dnl

dnl # Sendmail, Chap 24.9.19, Page 959
dnl # Turn on connection caching and set maximum number of simultaneous
dnl #  outbound connections kept open to 4; default is 2; this option also
dnl #  depends on MCI_CACHE_TIMEOUT (below)
define(`confMCI_CACHE_SIZE’,`4′)dnl

dnl # Sendmail, Chap 24.9.19, Page 959
dnl # Set time limit on how long a cached outbound connection may be
dnl #  kept open to 120 seconds (2 minutes) – see MCI_CACHE_SIZE above
define(`confMCI_CACHE_TIMEOUT’,`120s’)dnl

dnl # Sendmail, Chap 24.9.72, Page 1022
dnl # Disable time delay for queued messages not delivered on the first try
define(`confMIN_QUEUE_AGE’,`0′)dnl

dnl # Sendmail, Chapter 24.9.107, Page 1057
dnl # Disable MTA setting that forces MTA to queue each message and to sync
dnl #   to disk before forking (a system crash may result in lost mail)
define(`confSAFE_QUEUE’,`false’)dnl

dnl # Sendmail, Chap 24.9.109.13, Page 1065
dnl # Disable IDENT (RFC 1413) calls/turn off sending user-host verification
define(`confTO_IDENT’,`0′)dnl

dnl # Sendmail, Chap 24.9.120, Page 1077
dnl # Specify the maximum size, in bytes, of buffered xf* files (default is
dnl #   4096 bytes; 0 turns this off and is not recommended)
define(`confXF_BUFFER_SIZE’,`16384′)dnl

dnl # Sendmail, Chap 4.8.28, Page 192
dnl # Turn off E-Mail canonization (should be done by MSA, and this
dnl #     is a mail relay with no local users)
FEATURE(`nocanonify’)dnl

I’m assuming you’re using a modern version of sendmail – v8.12.10 or later. These settings may be different, or not exist at all, for older versions.

Source

If you would like to have a set of web pages that are protected, requiring a username/password to gain access, this tutorial will show you how to set it up. This is geared towards the Unix Apache httpd servers used on holly, lamar, and www.colostate.edu. If you are using another web server, you’ll need to check that server’s documentation to see how to do this.

Steps to Password-protect a Directory

First, create a subdirectory in your web area. For the sake of this tutorial, I have created the “protect” directory. Set the permissions on the directory so that the server has read/execute. I do this by using the local command chgrp-www to set the group to the www group. This is the group that the server runs under at Colorado State University for the lamar, holly and www servers. I have used the -sd flag which sets “set group id” for a directory. This will then force any files you create within the protect directory to the www group, so if you ftp files to this directory they will be automatically readable by the server but not by any other user on the system. I then cd into the protect directory.

cd ~ric/public_html
mkdir protect
chmod g+r,g+x,o-r,o-x protect
chgrp-www -sd protect
cd protect

Next you must create a .htaccess file inside the directory you want protected. You can use either the vi or pico editors on the supported systems mentioned above or ftp the file to this directory. If you are new to unix or know little about vi then I suggest you use the pico editor or ftp the .htaccess file. The command to edit with pico is “pico .htaccess”. The .htaccess file should contain the following lines. The items in bold are things you will want to change depending on the location of the AuthUserFile and content of AuthName.

AuthUserFile /z/ric/secret/.htpasswd
AuthGroupFile /dev/null
AuthName "Ric's protected files"
AuthType Basic

<Limit GET>
require valid-user
</Limit>

The AuthName is what the user will see when they’re prompted for a password – something to the effect of “Enter the username for Ric’s Protected files”. The AuthUserFile is location of the password file and should be not accessible with a url on the server for security reasons. This is a full unix path and the permissions should be set up like the “protect” directory using the chmod and chgrp-www commands above so the only one that can read this file is the owner and the server. To get the full path of a directory, cd to that directory and enter the command “pwd” to print the working directory path.

Now you’ll have to set up the password file. You’ll need to use the htpasswd program. It is included with the Apache httpd server.

First cd to the directory that contains the password file. In this example the password file is called .htpasswd and is in the directory /z/ric/secret/ as indicated by the AuthUserFile file entry in the .htaccess file. For every username you want to add to the password file, enter the following. (the -c is only required the first time; it indicates that you want to create the .htpasswd file).

$~ cd
$~ mkdir secret
$~ cd secret
$~ htpasswd -c .htpasswd pumpkin

[ you're prompted for the password for pumpkin]
[ if you have other users enter the following. Don't use the -c]

$~ htpasswd .htpasswd user2
$~ htpasswd .htpasswd user3

Again, make sure the permissions are set up like the “protect” directory using the chmod and chgrp-www commands above so the only one that can read files in the “secret” directory is the owner and the server.

Here is the protected page using the above setup to password protect this page. The username is “pumpkin” and password is “pie”.

[source]

If you’re going to be doing a lot of Geotargeting or IP Address Lookups, please take a feed instead which will preserve both our bandwidth and your bandwidth.

Simple GET

That said, there is an easy HTTP oriented API to locate IP addresses and Geocode them. If you don’t supply the “?ip=aa.bb.cc.dd” bit, then the ip address lookup of the calling machine will be located instead (here, the aa,bb,cc,dd are decimal digits). If you add &position=true to the end of the URL then latitude and longitude will be returned also. Both HTML and XML formats are supplied for your convenience.

http://api.hostip.info/country.php
US

http://api.hostip.info/get_html.php?ip=12.215.42.19
Country: UNITED STATES (US)
City: Sugar Grove, IL

http://api.hostip.info/get_html.php?ip=12.215.42.19&position=true
Country: UNITED STATES (US)
City: Sugar Grove, IL
Latitude: 41.7696
Longitude: -88.4588

http://api.hostip.info/?ip=12.215.42.19
[use the URL above for an example - XML too long to paste below]

Country Flag

Paste the following code into your HTML to get a country flag of the ip address. The database is significantly more accurate (it ought to be 100%) for countries than for cities. It would be nice if y’all would make the flag a link to the www.hostip.info home page (http://www.hostip.info/) so they can come by if they’re interested – it’ll only benefit you in the long run. After all, the results get more accurate as more visitors submit their IP addresses!
Flag of visitor’s location:

<A HREF=”http://www.hostip.info”>
<IMG SRC=”http://api.hostip.info/flag.php” BORDER=”0″ ALT=”IP Address Lookup”>
</A>

Flag of any IP address:

<A HREF=”http://www.hostip.info”>
<IMG SRC=”http://api.hostip.info/flag.php?ip=12.215.42.19″ ALT=”IP Address Lookup”>
</A>

Embedded Applet

The following is designed to be embedded within another HTML page using the OBJECT tag. This will reproduce the zoom-in applet, (or an explanatory message with a link to fix, if the IP address lookup is unknown). Which means you can embed the applet in your own site without needing to have the local database and map data (which runs to a few gigabytes…)

All you need do is include the OBJECT block below in your HTML. Note, you can also add “?ip=aaa.bbb.ccc.ddd” to the frame.html url below to map a specific IP address.

<OBJECT DATA='http://www.hostip.info/map/frame.html'

  TYPE='text/html' BORDER=0

  WIDTH=610 HEIGHT=330 HSPACE=0 VSPACE=0>

</OBJECT>

*NIX Shell Script

You can use the following shell script to call in your favorite *NIX environment.

#!/bin/bash
lynx -dump “http://api.hostip.info/get_html.php?ip=$1″

Source: [click here]

Max 30 mins work.

However, life is rarely so simple.
yum search dkim didn't find anything.

So, based on what I could find, I ended up here. Downloaded dkim-filter
2.4.1 and went on an epic voyage of discovery into the RFCs and other stuff.
I just want to install, configure and run the thing!

Anyway. I thought compilation would be straightforward, but no. More
unfamiliar stuff to read. I dutifully read the site.config.m4.dist, copied
to devtools/Site/site.config.m4 and hoped to make some intelligent decisions
on what options to enable.

 # ./Build
...
>Making all in:
>/etc/mail/dkim/dkim-milter-2.4.1/dkim-filter
>Configuration: pfx=, os=Linux, rel=2.6.23.1-10.fc7, rbase=2,
>rroot=2.6.23.1-10, arch=x86_64, sfx=, variant=optimized
>Using M4=/usr/bin/m4
>Creating
>/etc/mail/dkim/dkim-milter-2.4.1/obj.Linux.2.6.23.1-10.fc7.x86_64/dkim-filter
>using /etc/mail/dkim/dkim-milter-2.4.1/devtools/OS/Linux
>Making dependencies in
>/etc/mail/dkim/dkim-milter-2.4.1/obj.Linux.2.6.23.1-10.fc7.x86_64/dkim-filter
>make[1]: Entering directory
>`/etc/mail/dkim/dkim-milter-2.4.1/obj.Linux.2.6.23.1-10.fc7.x86_64/dkim-filter'
>rm -f sm_os.h
>ln -f -s ../../include/sm/os/sm_os_linux.h sm_os.h
>cc -M -I. -I../../include  -I../libdkim/   -D_REENTRANT config.c dkim-ar.c
>dkim-filter.c stats.c test.c util.c   dkim-testkey.c   dkim-testssp.c    >>
>Makefile
>In file included from config.h:23,
>                 from config.c:20:
>dkim-filter.h:22:29: error: libmilter/mfapi.h: No such file or directory
>In file included from dkim-ar.h:19,
>                 from dkim-ar.c:23:
>dkim-filter.h:22:29: error: libmilter/mfapi.h: No such file or directory
>dkim-filter.c:59:29: error: libmilter/mfapi.h: No such file or directory
>In file included from config.h:23,
>                 from dkim-filter.c:78:
>dkim-filter.h:22:29: error: libmilter/mfapi.h: No such file or directory
>In file included from test.c:31:
>test.h:24:29: error: libmilter/mfapi.h: No such file or directory
>In file included from util.c:49:
>dkim-filter.h:22:29: error: libmilter/mfapi.h: No such file or directory
>make[1]: *** [depend] Error 1
>make[1]: Leaving directory
>`/etc/mail/dkim/dkim-milter-2.4.1/obj.Linux.2.6.23.1-10.fc7.x86_64/dkim-filter'
>Making in
>/etc/mail/dkim/dkim-milter-2.4.1/obj.Linux.2.6.23.1-10.fc7.x86_64/dkim-filter
>make[1]: Entering directory
>`/etc/mail/dkim/dkim-milter-2.4.1/obj.Linux.2.6.23.1-10.fc7.x86_64/dkim-filter'
>cc -O2 -I. -I../../include  -I../libdkim/   -D_REENTRANT -DXP_MT   -c -o
>config.o config.c
>In file included from config.h:23,
>                 from config.c:20:
>dkim-filter.h:22:29: error: libmilter/mfapi.h: No such file or directory
>In file included from config.h:23,
>                 from config.c:20:
>dkim-filter.h:86: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’
>before ‘mlfi_connect’
>dkim-filter.h:87: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’
>before ‘mlfi_envfrom’
>dkim-filter.h:88: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’
>before ‘mlfi_header’
>dkim-filter.h:89: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’
>before ‘mlfi_eoh’
>dkim-filter.h:90: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’
>before ‘mlfi_body’
>dkim-filter.h:91: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’
>before ‘mlfi_eom’
>dkim-filter.h:92: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’
>before ‘mlfi_abort’
>dkim-filter.h:93: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’
>before ‘mlfi_close’
>make[1]: *** [config.o] Error 1
>make[1]: Leaving directory
>`/etc/mail/dkim/dkim-milter-2.4.1/obj.Linux.2.6.23.1-10.fc7.x86_64/dkim-filter'
>make: *** [all] Error 2

After some googling, a "yum install sendmail-devel" fixed this problem, and
a ./Build -c completed successfully.
I copied /devtools/OS/Linux to /devtools/Site/site.Linux.m4

./Build install was successful after manually creating dirs /usr/man/man15
and /usr/man/man18
Fedora manuals are in /usr/share/man
The files /usr/bin/dk* should have ownership root:root instead of bin.

Sendmail of Fedora 7 is currently 8.14.1:
# sendmail -d0.1
Version 8.14.1
Compiled with: DNSMAP HESIOD HES_GETMAILHOST LDAPMAP LOG MAP_REGEX
 MATCHGECOS MILTER MIME7TO8 MIME8TO7 NAMED_BIND NETINET NETINET6
 NETUNIX NEWDB NIS PIPELINING SASLv2 SCANF SOCKETMAP STARTTLS
 TCPWRAPPERS USERDB USE_LDAP_INIT

I created the keys, updated the dns zone files and decided to use user smmsp
instead of creating yet another user.

I created:
/var/db/dkim :
-rw-r----- 1 smmsp smmsp 887 2008-01-01 08:30 jan2008.admin.key.pem
-rw-r--r-- 1 smmsp smmsp 272 2008-01-01 08:30 jan2008.admin.public.pem

/var/run :
drwxr-xr-x 2 smmsp   smmsp   4096 2008-01-04 09:23 milter

and created this basic start/stop init script:
/etc/init.d/dkim-filter
then:
chkconfig --add dkim-filter
chkconfig dkim-filter on

contents:
>#
># dkim-filter        Starts /usr/bin/dkim-filter
>#
># chkconfig: 2345 67 33
>#
># description: Domain Keys Milter
># processname: dkim-filter
>#
># Source function library.
>. /etc/init.d/functions
>
>[ -f /usr/bin/dkim-filter ] || exit 0
>RETVAL=0
>
>umask 077
>
>start() {
>        echo -n $"Starting dkim-filter: "
>        /usr/bin/dkim-filter -x /etc/mail/dkim.conf
>        RETVAL=$?
>        if [ $RETVAL -eq 0 ]
>        then
>                echo_success
>                touch /var/lock/subsys/dkim-filter
>        else
>                echo_failure
>        fi
>        echo
>}
>stop() {
>        echo -n $"Shutting down dkim-filter: "
>        /bin/kill `cat /var/run/milter/dkim-filter.pid 2> /dev/null ` >
> /dev/null 2>&1
>        RETVAL=$?
>        sleep 3
>        if [ $RETVAL -eq 0 ]
>        then
>                echo_success
>                rm -f /var/lock/subsys/dkim-filter
>                rm -f /var/run/milter/dkim-filter.pid
>        else
>                echo_failure
>        fi
>        echo
>}
>rhstatus() {
>        status dkim-filter
>}
>restart() {
>        stop
>        start
>}
>
>case "$1" in
>  start)
>        start
>        ;;
>  stop)
>        stop
>        ;;
>  status)
>        rhstatus
>        ;;
>  restart|reload)
>        restart
>        ;;
>  condrestart)
>        [ -f /var/lock/subsys/dkim-filter ] && restart || :
>        ;;
>  *)
>        echo $"Usage: $0 {start|stop|status|restart|condrestart}"
>        exit 1
>esac
>
>exit $?

Now for configuration files:

/etc/mail/dkim.conf :
Canonicalization        relaxed/simple
Domain                  /etc/mail/domains
KeyFile                 /var/db/dkim/jan2008.admin.key.pem
#MTA                    MTA
Selector                jan2008.admin
SignatureAlgorithm      rsa-sha256
Socket                  inet:[EMAIL PROTECTED]
#Socket                 /var/run/milter/dkim-filter.sock
Syslog                  Yes
SyslogSuccess           Yes
Userid                  smmsp
PidFile                 /var/run/milter/dkim-filter.pid
SubDomains              Yes
X-Header                No
SendReports             No

/etc/mail/domains contains just one domain on one line.

and added to sendmail.rc:
INPUT_MAIL_FILTER(`dkim-filter', `S=inet:[EMAIL PROTECTED]')

I started the script with
/etc/init.d/dkim-filter start
and it worked, eg:
>Jan  4 10:58:10 gaia dkim-filter[6033]: Sendmail DKIM Filter v2.4.1 starting
>(args: -x /etc/mail/dkim.conf)

It even adds signatures to my messages (hopefully to this one), but silently
crashes regularly without any indication on processing a simple locally
generated mail from a perl script and/or/exor from logwatch or virus
notification from MailScanner. eg:

DKIMDEBUG=ct :
>Jan  3 02:57:18 gaia sendmail[12260]: m031vIL6012260: from=<[EMAIL
>PROTECTED]>, size=1780,, nrcpts=1, msgid=<[EMAIL PROTECTED]>,
>proto=ESMTP, daemon=MTA, relay=localhost [127.0.0.1]
>Jan  3 02:57:18 gaia dkim-filter[6926]: thread 0x41e02950 header
>Jan  3 02:57:18 gaia last message repeated 6 times
>Jan  3 02:57:18 gaia dkim-filter[6926]: thread 0x41e02950 eoh
>Jan  3 02:57:18 gaia sendmail[12260]: m031vIL6012260:
>milter_sys_read(dkim-filter): cmd read returned 0, expecting 5
>Jan  3 02:57:18 gaia sendmail[12260]: m031vIL6012260: Milter (dkim-filter): to
>error state
>Jan  3 02:57:18 gaia sendmail[12260]: m031vIL6012260: to=<[EMAIL PROTECTED]>,
>delay=00:00:00, mailer=esmtp, pri=31780, stat=queued

I have spent the last couple of days trying to solve this
The only relevant information I found was Jim Hermann's useful message and
thread last month
http://www.mail-archive.com/dkim-milter-discuss@lists.sourceforge.net/msg00409.html

I'm disappointed, disillusioned and frustrated in trying to nail jelly to a
wall... This doesn't say anything useful at all!
>milter_sys_read(dkim-filter): cmd read returned 0, expecting 5

It only seems to happen by locally generated mail, sometimes it even seemed
as if having a Reply-To: field influenced its crash frequency, but without
real diagnostic tools, skills and a lot of time, I can't solve it. I'm an
experienced sysadmin, not a C programmer! Programmers should try to make all
our lives easier!  

I want to get this working reliably and dependably on a few production
systems, and know what options to compile with and what settings to use for
Fedora, but I'm now stumped.

When it does work, another gripe is this padding too short error, which may
or may not be a reason for the verification failure:
>Jan  4 08:14:35 gaia dkim-filter[8389]: m047EY6O010080 SSL error:04067069:rsa
>routines:RSA_EAY_PUBLIC_DECRYPT:pkcs1 padding too short; error:04077068:rsa
>routines:RSA_verify:bad signature
>Jan  4 08:14:35 gaia dkim-filter[8389]: m047EY6O010080: bad signature data
>Jan  4 08:14:35 gaia sendmail[10080]: m047EY6O010080: Milter insert (1):
>header: Authentication-Results:  gaia.haveland.com; dkim=neutral (verification
>failed) [EMAIL PROTECTED]

How can a gmail signature fail verification? What did it fail on? What is
the "i" in  "header.i" ?
It was a mysql mailing list, so perhaps other headers got in the way, but
this isn't what I would call a robust solution! Omitheaders command in
dkim.conf seems to be a blanket fudge.

If we are to stand a chance of defeating spammers, then we have to make DKIM
easier to install and configure so mere mortals can install and use it, and
encourage adoption.  I'm sure many would like to see dkim-filter available
in rpm for various distros.

However, Network Solutions, amongst others need to wake up and allow people
to modify their DNS TXT attributes... Here's what their completely
ridiculous FAQ says on the subject:
http://customersupport.networksolutions.com/article.php?id=369

>"Can I Make Changes To The TXT Record
>
> Network Solutions does not currently support changes to the
> TXT record for a domain name registration.
>
> The TXT Record is strictly informational, not functional."

What planet are they living on?

Cheers,
Andy.

By default, Apache comes preconfigured to serve a maximum of 256 clients simultaneously. This particular configuration setting can be found in the file /etc/httpd/conf/httpd.conf

If your server has 2 GB of RAM, and you’re sharing your server with MySQL(true in my case), you’ll want to reserve about half of it for Apache (1 GB)

MaxClients: here is the process of determining MaxClients. type

ps -U apache -u apache u

See the number of apache process running in you command prompt.

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
apache 7694 0.0 0.3 42704 6680 ? S 18:30 0:00 /usr/sbin/httpd

The above indicates that a single httpd process is using 6.6 MB of RSS (Resident Set Size) memory (or non-swapped physical memory) and that it is using 42 MB of VSZ (Virtual Size) memory. This depends on the number of modules you have loaded and running in Apache.

As shared libraries are included in this number, it’s not 100 percent accurate. We can assume that half the RSS number is “real” memory. Let’s assume that each httpd process is using (6.6/2=3.3) 4 MB of memory. So if you have 1 GB ram then divide it with 4 MB of memory, which leaves room for around 256 concurrent httpd processes.

Set MaxClients 256

Or

Somebody prefers to set MaxClients using following rule

MaxClients = 150 x RAM (GB)

So for example if you have 2 GB RAM (dedicated for apache) set this value to 300. In my case IT WILL BE 150

Or

Some individuals maintain that each httpd thread uses about 5 MB of “real” memory. So they determine by the following way..

Or

MaxClients = RAM(MB)/5

So for example if you have 2 GB RAM (dedicated for apache) set this value to 409. In my case IT WILL BE 204(1 GB for apache)

Note: There is no reason for you to set it any higher unless you have a specific problem with this value. A high value can lead to a complete server hang in case of a DOS attack. A value too low can create timeout problems for your clients if the limit is reached

StartServers – Sets the number of child server processes created on startup. This setting depends greatly on the type of webserver you run. If you run low traffic websites on that server set it low to something like 5. If you have resource intensive websites on that server you should set it close to MaxClients.

MaxRequestsPerChild – Controls the number of request the a child serves before the child is killed. This should not be set too low as it will put an unnecessary load on the apache server to recreate the child. I suggest setting it to 1000.

But we are going to use 2000 for handling heavy traffic load properly.

MinSpareServers and MaxSpareServers – MaxSpareServers and MinSpareServers control how many spare (unused) child-processes Apache will keep alive while waiting for more requests to put them to use. Each child-process consumes resources, so having MaxSpareServers set too high can cause resource problems. On the other hand, if the number of unused servers drops below MinSpareServers, Apache will fork. Leave those values to: MinSpareServers 5 MaxSpareServers 10

ServerLimit: Its better to keep Server limit same as the value of MaxClients.

MaxRequestsPerChild: I’ve Kept default apache value for this one.

So few changes need to be made in httpd.conf file which is located in /etc/httpd/conf/ directory

<IfModule prefork.c>
StartServers     140
MinSpareServers    5
MaxSpareServers   10
ServerLimit      150
MaxClients       150
MaxRequestsPerChild  4000
</IfModule>

[Note]: Response time depends on MaxClients. If you increase the MaxClients number, server will response more quickly for each request but  a high value can lead to a complete server hang.

Ab is a tool for benchmarking the performance of your Apache HyperText Transfer Protocol (HTTP) server. It does this by giving you an indication of how many requests per second your Apache installation can serve.

uptime command in your root login should not yield a load average above 1, and the server should respond to commands quickly

ab -n 10000 -c 200 -k http://your_url
-c = concurrent connections
-t = time limit
-n = # of requests

Keep tuning until you hit your maximum desired load average. For servers used interactively often, having a load above 3 is way too much to use the server comfortably. For servers used mostly as real servers, a maximum load average of 10 should be acceptable. More than that, and you’ll find yourself needing to reboot the server when experiencing heavy traffic conditions, because no terminal or remote console will respond quickly to commands, and managing the server will be impossible.

How to configure few things in php.ini file for supporting huge traffic

* Enable the compression of HTML by putting in your php.ini:

output_handler = ob_gzhandler

** Switch from file based sessions to shared memory sessions. Compile PHP with the –with-mm option and

set session.save_handler=mm

Configure mysql. Change my.cnf file for better performance.

The database parameters are tuned for systems with 1 GB RAM (for ISO CD images). If you have higher RAM, please change the following in the “my.cnf” MySQL configuration file under /etc/mysql or /etc directory.

For a machine running with 512 MB of RAM, you can set these to:

key_buffer=128M table_cache=1024 sort_buffer=64M read_buffer=2M record_buffer=4M

For a machine running with 1 GB of RAM, you can set these to:

key_buffer=256M table_cache=2048 sort_buffer=128M read_buffer=2M record_buffer=8M

For a machine running with 2 GB of RAM, you can set these to:

key_buffer=512M table_cache=3072 sort_buffer=256M read_buffer=2M record_buffer=8M

For a machine running with 4 GB of RAM, you can set these to:

key_buffer=1G table_cache=4096 sort_buffer=512M read_buffer=2M record_buffer=8M

Original Post

To search in the current directory and all sub directories for a file named httpd.conf

find . -name “httpd.conf” -print

To find some string or text, type

find . -exec grep “MaxClients” ‘{}’ \; -print

This command will search in the current directory and all sub directories. All files that contain the string with the path.

If you want to just find each file then pass it on for processing use the -q grep option. This finds the first occurrance of the search string. It then signals success to find and find continues searching for more files.

find . -exec grep -q “www.athabasca” ‘{}’ \; -print

Send 1000 Request to apache using apache benchmark

ab -n 1000 -c 200 -k YOUR_URL

To view error log of httpd. type

grep -i maxclient /var/log/httpd/error_log*

To view Process status type and load average type top and uptime respectively.

To open a file and search something(Here Example is: MaxClients) from there type

vi +/MaxClients /etc/httpd/conf/httpd.conf

To view total memory used by httpd, type

ps -ylC httpd –sort:rss

Original Post

How to get help with svn?

If you are looking for svn reference in man pages, you have gone to the wrong place. To check the references of svn commands, simple do this:

svn help

This will make svn list all the available functions, to get the function reference, let say checkout

svn help checkout

The same thing goes to other svn related commands, such as svnadmin

svnadmin help

How to create a svn repository?

First of all what is repository? It is a core file for svn, or you can call it a centralized svn backup database. After created it, it is just a directory with its files. IMPORTANT! Do NOT try to modify or add something into the repository, unless you know what are you doing.

To create a svn repo, let say I wanna create a repo to store all my programming codes, I do this

svnadmin create /home/mysurface/repo/programming_repo

Remember try to use absolute path for everything, sometimes the relative path is not going to work.

How to import my existing directories into the new repo?

svn import /home/mysurface/programming file:///home/mysurface/repo/programming_repo -m “Initial import”

-m stand for log message, the first revision was created with log as “Initial import”. You need to specified URL for the repo, URL is the standard argument for svn. Therefore for local file, you need to specified with file://

How to see what is inside the repo?

svn list file:///home/mysurface/repo/programming_repo

Another way of listing all the files and folder in the tree view, I use svnlook

svnlook tree programming_repo

The difference between svn list and svnlook tree is one expect URL another one do not.

How to Install FFmpeg in Linux ~The Easy Way~

Original Post

FFmpeg is so important if you are planning to run a video website with streaming with conversion of video files to different video formats. This tutorial is intended for Centos/Redhat versions of Linux where any novice user can install ffmpeg without compiling the source which is a more traditional way of installing the FFmpeg software on linux servers. In this tutorial i will show you the easy way to install ffmpeg and ffmpeg-php (php extension) with just yum rather than compiling ffmpeg from source files.

FFmpeg (http://ffmpeg.mplayerhq.hu)
Mplayer + Mencoder (http://www.mplayerhq.hu/design7/dload.html)
Flv2tool (http://inlet-media.de/flvtool2)
Libogg + Libvorbis (http://www.xiph.org/downloads)
LAME MP3 Encoder (http://lame.sourceforge.net)
FlowPlayer – A Free Flash Video Player – http://flowplayer.org/

Installing FFMpeg

yum install ffmpeg ffmpeg-devel

If you get package not found, then you will need to add few lines in the yum repository for dag packages installation. Create a file named dag.repo in /etc/yum.repos.d with the following contents on it

[dag]
name=Dag RPM Repository for Red Hat Enterprise Linux
baseurl=http://apt.sw.be/redhat/el$releasever/en/$basearch/dag
gpgcheck=1
enabled=1

then

yum install ffmpeg ffmpeg-devel

If everything is fine, then the installation should proceed smoothly. If not you will get something like warning GPG public key missing .

Common Errors

To fix rpmforge GPG key warning:

rpm -Uhv http://apt.sw.be/redhat/el5/en/i386/rpmforge/RPMS/rpmforge-release-0.3.6-1.el5.rf.i386.rpm

For more information refer to this faq depending on Centos version

Missing Dependency Error:

If you get missing dependency error like shown below, in the middle of ffmpeg installation

Error: Missing Dependency: libc.so.6(GLIBC_2.4) is needed by package ffmpeg
Error: Missing Dependency: libtheora.so.0(libtheora.so.1.0) is needed by package ffmpeg
Error: Missing Dependency: rtld(GNU_HASH) is needed by package ffmpeg
Error: Missing Dependency: libc.so.6(GLIBC_2.4) is needed by package imlib2
Error: Missing Dependency: rtld(GNU_HASH) is needed by package a52dec
Error: Missing Dependency: rtld(GNU_HASH) is needed by package imlib2
Error: Missing Dependency: rtld(GNU_HASH) is needed by package gsm
Error: Missing Dependency: libc.so.6(GLIBC_2.4) is needed by package x264
Error: Missing Dependency: rtld(GNU_HASH) is needed by package xvidcore
Error: Missing Dependency: libc.so.6(GLIBC_2.4) is needed by package lame
Error: Missing Dependency: libc.so.6(GLIBC_2.4) is needed by package a52dec
Error: Missing Dependency: rtld(GNU_HASH) is needed by package faad2
Error: Missing Dependency: rtld(GNU_HASH) is needed by package x264
Error: Missing Dependency: rtld(GNU_HASH) is needed by package lame
Error: Missing Dependency: libc.so.6(GLIBC_2.4) is needed by package xvidcore
Error: Missing Dependency: libc.so.6(GLIBC_2.4) is needed by package faac
Error: Missing Dependency: libc.so.6(GLIBC_2.4) is needed by package faad2
Error: Missing Dependency: libgif.so.4 is needed by package imlib2
Error: Missing Dependency: rtld(GNU_HASH) is needed by package faac
Error: Missing Dependency: libc.so.6(GLIBC_2.4) is needed by package gsm
Error: Missing Dependency: libpng12.so.0(PNG12_0) is needed by package imlib2
Error: Missing Dependency: rtld(GNU_HASH) is needed by package libmp4v2
Error: Missing Dependency: libc.so.6(GLIBC_2.4) is needed by package libmp4v2

then most commonly you have GLIB 2.3 installed instead of GLIB 2.4 version. To check the current GLIB version installed on your server. just use:

yum list glib*

and it should list the latest GLIB package version.

The reason i was getting this error was my rpmforge packages was pointed to centos 5 versions instead of centos 4.6.

To fix dependency error:

To fix this error, you might need to check your rpmforge packages compatible to the release of your existing CentOS version.
Check the file /etc/yum.repos.d/rpmforge.repo and it should look like for Centos 4.6(Final). If you have lines like http://apt.sw.be/redhat/el5/en/mirrors-rpmforge you might need to make changes to the rpmforge.repos like shown below

Note: Backup the original rpmforge.repo file before you edit its content.

[rpmforge]
name = Red Hat Enterprise $releasever – RPMforge.net – dag
#baseurl = http://apt.sw.be/redhat/el4/en/$basearch/dag
mirrorlist = http://apt.sw.be/redhat/el4/en/mirrors-rpmforge
#mirrorlist = file:///etc/yum.repos.d/mirrors-rpmforge
enabled = 1
protect = 0
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rpmforge-dag
gpgcheck = 1

To know what linux type and version you are running

cat /etc/redhat-release

Once this is done, do again:  yum install ffmpeg.

This trick resolved the problem in my linux box running Centos 4.6 and this is the only way i found to install ffmpeg using yum.

To check the FFmpeg working:

Finally, check the ffmpeg whether it is working or not.

> ffmpeg
> ffmpeg -formats
> ffmpeg –help
// This lists path of mpeg, its modules and other path information

ffmpeg -i Input.file Output.file

To check what audi/video formats are supported

ffmpeg -formats > ffmpeg-format.txt

Open the ffmpeg-formats.txt to see the ooutput

D means decode
E means encode
V means video
A means audio
T = Truncated

Install FFMPEG-PHP Extension

FFmpeg-php is a very good extension and wrapper for PHP which can pull useful information about video through API interface. Inorder to install it you will need to download the source file and then compile and install extension in your server. You can download the source tarball : http://ffmpeg-php.sourceforge.net/

wget /path/to/this/file/ffmpeg-php-0.5.2.1.tbz2

tar -xjf ffmpeg-0.5.2.1.tbz2

phpize

./configure
make
make install

Common Errors

1. If you get command not found error for phpize, then you will need to do yum install php-devel

2. If you get error like “ffmpeg headers not found” while configuring the source.

configure: error: ffmpeg headers not found. Make sure ffmpeg is compiled as shared libraries using the –enable-shared option

then it means you have not installed ffmpeg-devel packages.

To Fix: Just install ffmpeg-devel using

yum install ffmpeg-devel

3. If you get an error like shared libraries not found problem and the program halts in the middle, then you must specify the ffmpeg installed path explicitly to the ./configure.

configure: error: ffmpeg shared libraries not found. Make sure ffmpeg is compiled as shared libraries using the –enable-shared option

To Fix:

1. First find out the ffmpeg path with ffmpeg –help command. The prefix default path should be like /usr/local/cpffmpeg
2. Configure the FFmpeg-php with –with-ffmpeg option

./configure –with-ffmpeg=/usr/local/cpffmpeg

That should resolve the problem!

Editing PHP.INI

Once you have done that without any problems then you will see the php extension file /usr/local/lib/php/extensions/no-debug-non-zts-20060613/ffmpeg.so and you will need mention that extension in php.ini file

nano /usr/local/lib/php.ini

Put the below two lines at the end of the php.ini file

[ffmpeg]
extension=ffmpeg.so

Then restart the server service httpd restart

To check whether ffmpeg enabled with php, point your browser to test.php file. It should show the confirmation of installed ffmpeg php extension

// #test.php

<?php

phpinfo()

?>

If any case the ffmpeg does not show in the phpinfo() test make sure that php.ini path to ffmpeg.so is correct. Still the problem occurs, the reason could be you might be using older versions of ffmpeg-php which is buggy. Just download the latest version of ffmpeg-php source then compile it.

Installing Mplayer + Mencoder

Just issue the following yum commands to install the rest of the packages.

yum install mplayer mencoder

Installing FlvTool2

Flvtool2 is a flash video file manipulation tool. It can calculate metadata and can cut and edit cue points for flv files.

If you are on Centos 5 try yum install flvtool2 with dag repository and if you get package not found you will need to manually download and compile the flvtool2. You can download latest version of flvtool2 here: http://rubyforge.org/projects/flvtool2/

wget <url-link>

ruby setup.rb config
ruby setup.rb setup
sudo ruby setup.rb install

If you get command not found error, it probably means that you dont have ruby installed.

yum install ruby

Thats it! Once ffmpeg works fine with php extension, download a sample video, convert to .flv format in the command line and plug it to flowplayer to see it work on your web browser. Try also to download the video file offline and see whether the converted flv file works well with both audio and video.

Useful Links

• FFmpeg (http://ffmpeg.mplayerhq.hu)
• Mplayer + Mencoder (http://www.mplayerhq.hu/design7/dload.html)
• Flv2tool (http://inlet-media.de/flvtool2)
• Libogg + Libvorbis (http://www.xiph.org/downloads)
• LAME MP3 Encoder (http://lame.sourceforge.net)
• FlowPlayer – A Free Flash Video Player – http://flowplayer.org/
• Install FFmpeg from Compiling Source (Tutorial Link)
• Nice FFmpeg Installation Tutorial (click here)
• Important Audio Codecs (http://www.mplayerhq.hu/DOCS/HTML/en/audio-codecs.html)
• Common Errors & Fixes while Installing FFmpeg (click here)

/usr/bin/ld: cannot find -lltdl
collect2: ld returned 1 exit status

What you need to do, is just follow the below steps.

1. Verify that the libtool and libtool-ltdl packages are installed.
2. Symlink libltdl.so to libltdl.so.x.x.x

If libtool and libtool-ltdl already exist, you may go to Step Two.
Step One

[root@banzaibill ~]# yum install libtool-ltdl libtool

Now you have libtool installed. To check it out, do:

[root@banzaibill ~]# yum info libtool*

Step Two

PHP looks for the libltdl library only at /usr/lib/libltdl.so

The symlink to this file is not included in the libtool packages. Do below commands:

[root@banzaibill ~]# cd /usr/lib
[root@banzaibill lib]# ln -s libltdl.so.3.1.4 libltdl.so

And that’s it. PHP should configure and compile without error.

I found good examples for this.

- Quick HOWTO (from LinuxHomeNetworking.com) – download

- Sample IPTABLES Configuration (RedHat/CentOS) – download

When you’re running REDHAT/CENTOS platform for your server, sometimes the RPM wont run normally or become stuck. This caused by the RPM database not properly builded or been corrupted.

Errors would be like :

rpmdb: Program version 4.3 doesn’t match environment version
error: db4 error(-30974) from dbenv->open: DB_VERSION_MISMATCH: Database environment version mismatch
error: cannot open Packages index using db3 – (-30974)
error: cannot open Packages database in /var/lib/rpm

rpmdb: Program version 4.3 doesn’t match environment version
error: db4 error(-30974) from dbenv->open: DB_VERSION_MISMATCH: Database environment version mismatch
error: cannot open Packages database in /var/lib/rpm

So, to solve those issues,
you can run these commands from ROOT access:

rm -f /var/lib/rpm/__db*
rpm -vv –rebuilddb

The ‘-vv’ parameter on rpm, will STDOUT.

To run it as background process, do:

rpm –rebuilddb &

I declare from the beginning that I am no authority on digital certificates.

This document is a summary of all the articles I have read about openssl. It describes in short how to become your own Certificate Authority (CA) and how to create and sign your own certificate requests. Make no mistake, these certificates are good only for personal use or for use in your intranet in order to provide a secure way to login or communicate with your services, so that passwords or other data is not transmitted in the clear. Noone else will or should trust these certificates.

Prerequisites

The package openssl should be installed in the machine you will use to manage your certificates or create the certificate requests.

First things first…

The openssl package comes with some scripts that can help you create your server certificates fast, but here I will describe how to set things up from scratch in a new directory, so that you can customize things later if you like or delete everything without touching openssl’s or the system’s default files. This article is based on a Fedora installation, but will do for all distributions.

Creating the necessary directories

First of all we will create a directory tree where all certificate stuff will be kept. Fedora’s default directory is /etc/pki/tls/. So, as root, we create our own directories:

# mkdir -m 0755 /etc/pki_jungle

And then we create our CA’s directory tree:

# mkdir -m 0755 \
     /etc/pki_jungle/myCA \
     /etc/pki_jungle/myCA/private \
     /etc/pki_jungle/myCA/certs \
     /etc/pki_jungle/myCA/newcerts \
     /etc/pki_jungle/myCA/crl

• myCA is our Certificate Authority’s directory.
• myCA/certs directory is where our server certificates will be placed.
• myCA/newcerts directory is where openssl puts the created certificates in PEM (unencrypted) format and in the form cert_serial_number.pem (eg 07.pem). Openssl needs this directory, so we create it.
• myCA/crl is where our certificate revokation list is placed.
• myCA/private is the directory where our private keys are placed. Be sure that you set restrictive permissions to all your private keys so that they can be read only by root, or the user with whose priviledges a server runs. If anyone steals your private keys, then things get really bad.

Initial openssl configuration

We are going to copy the default openssl configuration file (openssl.cnf) to our CA’s directory. In Fedora, this file exists in /etc/pki/tls. So, we copy it to our CA’s dir and name it openssl.my.cnf. As root:

# cp /etc/pki/tls/openssl.cnf /etc/pki_jungle/myCA/openssl.my.cnf

This file does not need to be world readable, so we change its attributes:

# chmod 0600 /etc/pki_jungle/myCA/openssl.my.cnf

We also need to create two other files. This file serves as a database for openssl:

# touch /etc/pki_jungle/myCA/index.txt

The following file contains the next certificate’s serial number. Since we have not created any certificates yet, we set it to “01“:

# echo '01' > /etc/pki_jungle/myCA/serial

Things to remember

Here is a small legend with file extensions we will use for the created files and their meaning. All files that will be created will have one of these extensions:

• KEY – Private key (Restrictive permissions should be set on this)
• CSR – Certificate Request (This will be signed by our CA in order to create the server certificates. Afterwards it is not needed and can be deleted)
• CRT – Certificate (This can be publicly distributed)
• PEM – We will use this extension for files that contain both the Key and the server Certificate (Some servers need this). Permissions should be restrictive on these files.
• CRL – Certificate Revokation List (This can be publicly distributed)

Create the CA Certificate and Key

Now, that all initial configuration is done, we may create a self-signed certificate, that will be used as our CA’s certificate. In other words, we will use this to sign other certificate requests.

Change to our CA’s directory. This is where we should issue all the openssl commands because here is our openssl’s configuration file (openssl.my.cnf). As root:

# cd /etc/pki_jungle/myCA/

And then create your CA’s Certificate and Private Key. As root:

# openssl req -config openssl.my.cnf -new -x509 -extensions v3_ca -keyout private/myca.key -out certs/myca.crt -days 1825

This creates a self-signed certificate with the default CA extensions which is valid for 5 years. You will be prompted for a passphrase for your CA’s private key. Be sure that you set a strong passphrase. Then you will need to provide some info about your CA. Fill in whatever you like. Here is an example:

Country Name (2 letter code) [GB]:GR
State or Province Name (full name) [Berkshire]:Greece
Locality Name (eg, city) [Newbury]:Thessaloniki
Organization Name (eg, company) [My Company Ltd]:My Network
Organizational Unit Name (eg, section) []:My Certificate Authority
Common Name (eg, your name or your server's hostname) []:server.example.com
Email Address []:whatever@server.example.com

Two files are created:

• certs/myca.crt – This is your CA’s certificate and can be publicly available and of course world readable.
• private/myca.key – This is your CA’s private key. Although it is protected with a passphrase you should restrict access to it, so that only root can read it:

  # chmod 0400 /etc/pki_jungle/myCA/private/myca.key

More openssl configuration (mandatory)

Because we use a custom directory for our certificates’ management, some modifications to /etc/pki_jungle/myCA/openssl.my.cnf are necessary. Open it in your favourite text editor as root and find the following part (around line 35):

[ CA_default ]

dir     = ../../CA      # Where everything is kept
certs       = $dir/certs        # Where the issued certs are kept
crl_dir     = $dir/crl      # Where the issued crl are kept
database    = $dir/index.txt    # database index file.
#unique_subject = no            # Set to 'no' to allow creation of
                    # several ctificates with same subject.
new_certs_dir   = $dir/newcerts     # default place for new certs.

certificate = $dir/cacert.pem   # The CA certificate
serial      = $dir/serial       # The current serial number
#crlnumber  = $dir/crlnumber    # the current crl number must be
                    # commented out to leave a V1 CRL
crl     = $dir/crl.pem      # The current CRL
private_key = $dir/private/cakey.pem    # The private key
RANDFILE    = $dir/private/.rand    # private random number file

x509_extensions = usr_cert      # The extentions to add to the cert

You should modify the following settings in order to coform to our custom directory and our custom CA key and certificate:

[ CA_default ]

dir     = .                # <--CHANGE THIS
certs       = $dir/certs
crl_dir     = $dir/crl
database    = $dir/index.txt
#unique_subject = no

new_certs_dir   = $dir/newcerts

certificate = $dir/certs/myca.crt   # <--CHANGE THIS
serial      = $dir/serial
#crlnumber  = $dir/crlnumber

crl     = $dir/crl.pem
private_key = $dir/private/myca.key    # <--CHANGE THIS
RANDFILE    = $dir/private/.rand

x509_extensions = usr_cert

Create a Server certificate

Further openssl.my.cnf file’s customization is possible, so that we define our policy for certificate creation and signing or define our desired extensions for the new certificates. I may add this info to a future version of this document. It’s easy though, just try to familiarize yourself with the openssl.cnf’s structure and you’ll figure it out.

Anyway, the certificates we are going to create, without customizing openssl.my.cnf any further, are general purpose certificates and their usage in not restricted to server authentication only. One thing that you should take a note of is that the private keys will not be protected by a passphrase, so that when the services are restarted they do not ask for a passphrase. This means that you should set restrictive permissions on the private keys, so that only root or the user under whose priviledges a server runs can read these files.

Generate a Certificate Request

First, we change to our CA’s directory:

# cd /etc/pki_jungle/myCA/

Then we create the certificate request:

# openssl req -config openssl.my.cnf -new -nodes -keyout private/server.key -out server.csr -days 365

The -nodes option is needed so that the private key is not protected with a passphrase. If you do not intend to use the certificate for server authentication, you should not include it in the above command.
You can customize the number of days you want this certificate to be valid for.

You will be prompted for the certificate’s info. Here is an example:

Country Name (2 letter code) [GB]:GR
State or Province Name (full name) [Berkshire]:Greece
Locality Name (eg, city) [Newbury]:Thessaloniki
Organization Name (eg, company) [My Company Ltd]:My Network
Organizational Unit Name (eg, section) []:My Web Server
Common Name (eg, your name or your server's hostname) []:www.server.example.com
Email Address []:whatever@server.example.com

The Common Name (CN) is the info that uniquely distinguishes your service, so be sure that you type it correctly.

When prompted for some extra attributes (challenge password, optional company name) just hit the [Enter] key.
Two files are created:

• server.csr – this is the certificate request.
• private/server.key – this is the private key, which is not protected with a passphrase.

Set restrictive permissions on the private key. Only root or the user that is used to run the server should be able to read it. For example:

# chown root.root /etc/pki_jungle/myCA/private/server.key
# chmod 0400 /etc/pki_jungle/myCA/private/server.key

Or:

# chown root.apache /etc/pki_jungle/myCA/private/server.key
# chmod 0440 /etc/pki_jungle/myCA/private/server.key

Sign the Certificate Request

Now we are going to sign the certificate request and generate the server’s certificate.

First, we change to our CA’s directory:

# cd /etc/pki_jungle/myCA/

Then we sign the certificate request:

# openssl ca -config openssl.my.cnf -policy policy_anything -out certs/server.crt -infiles server.csr

You will need to supply the CA’s private key in order to sign the request. You can check the openssl.my.cnf file about what policy_anything means. In short, the fields about the Country, State or City is not required to match those of your CA’s certificate.

After all this is done two new files are created:

• certs/server.crt – this is the server’s certificate, which can be made available publicly.
• newcerts/01.pem – This is exactly the same certificate, but with the certificate’s serial number as a filename. It is not needed.

You can now delete the certificate request (server.csr). It’s no longer needed:

# rm -f /etc/pki_jungle/myCA/server.csr

Verify the certificate

You can see the certificate’s info with the following:

# openssl x509 -subject -issuer -enddate -noout -in /etc/pki_jungle/myCA/certs/server.crt

Or the following:

# openssl x509 -in certs/server.crt -noout -text

And verify that the certificate is valid for server authentication with the following:

# openssl verify -purpose sslserver -CAfile /etc/pki_jungle/myCA/certs/myca.crt /etc/pki_jungle/myCA/certs/server.crt

Server certificate and key in one file

Some servers, for example vsftpd, require that both the private key and the certificate exist in the same file. In a situation like that just do the following:

# cat certs/server.crt private/server.key > private/server-key-cert.pem

You should restrict access to the final file and delete server.crt and server.key since thay are no longer needed.

# chown root.root private/server-key-cert.pem
# chmod 0400 private/server-key-cert.pem
# rm -f certs/server.crt
# rm -f private/server.key

Revoke a Server Certificate

If you do not want a certificate to be valid any more, you have to revoke it. This is done with the command:

# openssl ca -config openssl.my.cnf -revoke certs/server.crt

Then you should generate a new CRL (Certificate Revokation List):

# openssl ca -config openssl.my.cnf -gencrl -out crl/myca.crl

The CRL file is crl/myca.crl.

Distribute your certificates and CRL

Your CA’s certificate and your servers’ certificates should be distributed to those who trust you so they can import them in their client software (web browsers, ftp clients, email clients etc). The CRL should also be published.

There are often problems with sendmail once it has been installed due to the tightening up of sendmail to stop spammers

Sendmail-8.11.6-15 Connection refused

Sendmail & tcp wrapper rejection

Cannot relay from valid ip address (Outlook)

1) Sendmail-8.11.6-15 Connection refused

Cannot telnet to port 25, then Sendmail has not been corretly set up. This is a problem with RedHat 7.3 or more where Sendmail by default is set to only send from the localhost, you could say this is Good as Sendmail can not spew when set up on a system that is not going to use it.

File: /etc/sendmail.cf

Did you make the DAEMON_OPTIONS change mentioned in the release notes? Your sendmail.cf should *NOT* have this line:

O DaemonPortOptions=Port=smtp,Addr=127.0.0.1, Name=MTA

It needs to be hashed out to this:

#O DaemonPortOptions=Port=smtp,Addr=127.0.0.1, Name=MTA

File: /etc/mail/sendmail.mc

You can also change sendmail.mc, but this is just the configuration file that is used to create sendmail.cf. You can either delete it or change the .mc file from

DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA’)

to:

dnl DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA’)

You do not need to rebuild sendmail.cf if you make the changes directly to sendmail.cf.

To rebuild sendmail.cf is a headache so better to edit sendmail.cf and restart sendmail.

2) Sendmail & tcpwrappers rejection

File: /etc/hosts.allow

sendmail: ALL EXCEPT \
203.204. \
218.

3) Cannot relay from valid ip address (Outlook)

Sendmail has been installed and the above patches have been appilied, email is being sent fine from with in Horde, but as soon as a valid client (ip address) seeks to send emails through the server using a client mail program (outlook), we get a Relaying Rejected message.

The answer to this problem was archived when researching this page on sendmail.org.
http://www.sendmail.org/~ca/email/relayingdenied.html

Feb 24 08:39:20 mail sendmail[17602]: i1NJdKCq017602: ruleset=check_rcpt, arg1=<someone@someone.co.nz>, relay=me.somehereelse.co.nz [192.168.xx.19], reject=550 5.7.1 <someone@someone.co.nz>… Relaying denied
(parts of message changed for security)

Generally the /etc/mail/access file only has allowed client ip addresses for relaying. Now with new versions of Sendmail I have found it necessary to put in the allowed name that the PCs are giving to sendmail.

File: /etc/mail/access

access-info.co.nz                 RELAY

This lines is needed in /etc/mail/access to enable name resolution.

You may also need the following line in hosts to also enable dns ip lookup

File: /etc/hosts

192.168.xx.xx    laptop.access-info.co.nz    laptop

Replace xs with valid ip address for the PC trying to send via outlook.

By: Abe Getchell 2008-10-03

This article discusses the process of recovering deleted data from an ext3 partition, on a system running Linux, using a process called data carving. This basic technique is useful in any number of situations, such as recovering data that has been accidentally deleted by a user, information removed in an attempt to erase signs of a system intrusion that could be used to track the source, or data erased by an end-user attempting to cover up an acceptable use policy infraction.

This article assumes that you have a basic understanding of ext3 and the inner workings of filesystems. It is important to note that there is a certain amount of risk associated with this process. When performed improperly, the data you are attempting to recover, or other data stored on the system, could be permanently lost. While this technique is quite accurate most of the time, and very useful in any number of different situations, it is not “forensically sound” and will not hold up legally for use in court. Special software, hardware, and procedures — or professional services — are a must in situations when legal action is required.

The tools used in this article are freely available and can be downloaded from their respective websites.

The basic recovery process

In this section we will go step-by-step through the data recovery process and describe the tools, and their options, in detail. We start by listing a directory below.

[abe@abe-laptop test]$ ls -al
total 27
drwxrwxr-x 2 abe abe 4096 2008-03-29 17:48 .
drwx—— 71 abe abe 4096 2008-03-29 17:47 ..
-rwxr–r– 1 abe abe 42736 2008-03-29 17:47 weimaraner1.jpg

In the listing above we can see that there is a file named weimaraner1.jpg in the test directory. This is a picture of my dog. I don’t want to delete it. I like my dog.

[abe@abe-laptop test]$ rm -f *

Here we can see I am deleting it. Whoops! Sorry buddy. Let’s gather some basic information about the system so we can begin the recovery process.

[abe@abe-laptop test]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 71G 14G 53G 21% /
/dev/sda1 99M 19M 76M 20% /boot
tmpfs 1007M 12K 1007M 1% /dev/shm
/dev/sdb1 887M 152M 735M 18% /media/PUBLIC

Here we see that the full path to the test directory (which is /home/abe/test) is part of the / filesystem, represented by the device file /dev/sda2.

[abe@abe-laptop test]$ su -
Password:
[root@abe-laptop ~]# debugfs /dev/sda2

Using su to gain root access, we can start the debugfs program giving it the target of /dev/sda2. The debugfs program is an interactive file system debugger that is installed by default with most common Linux distributions. This program is used to manually examine and change the state of a filesystem. In our situation, we’re going to use this program to determine the inode which stored information about the deleted file and to what block group the deleted file belonged.

debugfs 1.40.4 (31-Dec-2007)
debugfs: cd /home/abe/test
debugfs: ls -d
1835327 (12) . 65538 (4084) .. <1835328> (4072) weimaraner1.jpg

After debugfs starts, we cd into /home/abe/test and run the ls -d command. This command shows us all deleted entries in the current directory. The output shows us that we have one deleted entry and that its inode number is 1835328 — that is, the number between the angular brackets.

debugfs: imap <1835328>
Inode 1835328 is part of block group 56
located at block 1835019, offset 0×0f80

The next command we want to run is imap, giving it the inode number above so we can determine to which block group the file belonged. We see by the output that it belonged to block group 56.

debugfs: stats
[...lots of output...]
Blocks per group: 32768
[...lots of output...]
debugfs: q

Running the stats command will generate a lot of output. The only data we are interested in from this list, however, is the number of blocks per group. In this case, and most cases, it’s 32768. Now we have enough data to be able to determine the specific set of blocks in which the data resided. We’re done with debugfs now, so we type q to quit.

[root@abe-laptop ~]# dls /dev/sda2 1835008-1867775 > /media/PUBLIC/block.dat

——————–

The next thing we need to do is pull all unallocated blocks from block group 56 so we can examine their content. The dls program, from The Sleuth Kit (TSK), allows us to do just that. We simply need to know the device file, a range of blocks, and have enough space in the appropriate place to output this data. Using the information above, we can calculate the block range by multiplying the block group number and the block group size and then multiplying the block group number plus one by the blocks per group minus one. In this case, the formula would look like this:

(56 x 32768) through ((56 + 1) x 32768 – 1)

This would give us a range of 1835008 through 1867775. It’s very important that the destination of the output does not reside on the same partition as the data you’re attempting to recover. What will most likely be a large amount of data being written to disk from the output of this command could potentially overwrite the data you are trying to recover (as the blocks which stored the data from the deleted file have already been marked unallocated). You want as little disk activity as possible on the partition you’re working with. In this example, I’m using a USB thumb drive (located on /media/PUBLIC) as a location to store this data.

[root@abe-laptop ~]# mkdir /media/PUBLIC/output
[root@abe-laptop ~]# foremost -dv -t jpg -i /media/PUBLIC/block.dat -o /media/PUBLIC/output/

Next we need to attempt to extract this data from the unallocated blocks we extracted with the dls command above. To do this, we are going to use Foremost. This program is used to recover files based on header information, footer information, and internal data structures. This is the process, mentioned earlier, called data carving. First we are going to create a directory to store the foremost output (again, this should be on a separate partition). Next we are going to run the foremost command giving it the file type of jpg (which is an internally recognized type – more on custom types below), the input file, and the output directory. The output from this command is listed below.

Foremost version 1.5.3 by Jesse Kornblum, Kris Kendall, and Nick Mikus
Audit File

Foremost started at Sat Mar 29 18:02:29 2008
Invocation: foremost -dv -t jpg -i /media/PUBLIC/block.dat -o /media/PUBLIC/output/
Output directory: /media/PUBLIC/output
Configuration file: /usr/local/etc/foremost.conf
Processing: /media/PUBLIC/block.dat
|——————————————————————
File: /media/PUBLIC/block.dat
Start: Sat Mar 29 18:02:29 2008
Length: 110 MB (115941376 bytes)

Num Name (bs=512) Size File Offset Comment

0: 00033272.jpg 26 KB 17035264
1: 00033328.jpg 184 KB 17063936
2: 00033704.jpg 58 KB 17256448
3: 00033824.jpg 62 KB 17317888

[...]

*46: 00210136.jpg 2 KB 107589632
47: 00210144.jpg 3 KB 107593728
48: 00210392.jpg 6 KB 107720704
*
Finish: Sat Mar 29 18:02:29 2008

49 FILES EXTRACTED

jpg:= 49
——————————————————————

Foremost finished at Sat Mar 29 18:02:29 2008
[root@abe-laptop ~]#

As we can see, Foremost found forty-nine previously deleted jpg files (this output is also saved in a file named audit.txt in the root of the specified output directory). How do we know which is the file we are trying to recover? We could, as is most commonly done, open all of these files and see their contents. Another option is to simply compare file sizes. We know from our directory listing above that the jpg file we are looking for is 41k in size. There’s only one file that foremost extracted into the output directory that’s 41k, and indeed, 00114144.jpg is the file we are attempting to recover. Comparing size only works, of course, if you “know your data”. Integrity checking programs such as Tripwire play a big role in a recovery operation as you can identify the recovered data without ever inspecting the content, as well as verify its integrity. This becomes quite useful if the information you’re attempting to recover is confidential and you are not authorized to view the data.

Defining custom types in Foremost

As of Foremost v1.5.3, the internally supported data types that the program will recover without custom rules are jpg, gif, png, bmp, avi, exe, mpg, wav, riff, wmv, mov, pdf, ole, doc, zip, rar, htm, and cpp. If you need to recover data beyond these built-in data types, you will need to define custom types in Foremost’s configuration file (foremost.conf).

———————-

An entry that defines a type in the foremost configuration file (as explained in the documentation at the beginning of foremost.conf or in the manpage) consists of several columns: extension, case sensitivity, maximum size, header and footer (optional), and special keywords (optional). As an example that most should be familiar with, here is the entry for an html file:

htm n 50000 <html </html>

We see here that the file extension is htm (NONE can be specified if no file extension should be used during the output of extracted data), the header and footer are not case sensitive, the maximum file size is 50k bytes (which means that 50k bytes after the header will be recovered if no footer is specified or 50k bytes will be recovered if that amount of data is recovered before the defined footer is detected), the recovered file should start with “<html” (header) and end with “</html>” (footer).

The ASCII keyword can also be used when attempting to recover ASCII files. Specifying this keyword at the end of an entry will tell Foremost to extract all ASCII printable characters before and after the keyword defined. An example of this would be a type to recover a perl script. If, for example, you need to recover a perl script that you know included Crypt::CBC, you could use the following type definition:

pl y 100000 Crypt::CBC Crypt::CBC ASCII

Note that Crypt::CBC is listed in both the header and footer fields. This is done so that Foremost will recognize this as the string to search around when the ASCII keyword is used. A more general type to find perl scripts could be defined as follows:

pl n 100000 #!/usr/bin/perl #!/usr/bin/perl ASCII

When attempting to recover files that are not ASCII, hexadecimal and octal notation can be used by specifying \x[0-f][0-f] or \[0-3][0-7][0-7], respectively. Below is an example of hexadecimal notation describing the header and footers of a gif file:

gif y 155000000 \x47\x49\x46\x38\x37\x61 \x00\x3b

As you may have realized by now, Foremost is a very powerful tool. Learn its intricacies and it can be a wonderfully flexible tool in data recovery and computer security forensic operations. Read the Foremost man page or consult the configuration file for a complete guide to creating custom data types.

ext2 vs ext3 Data Recover

You may be asking yourself why this process is so much more difficult with ext3 than it is with ext2? This question is answered by one of the ext3 developers in the Linux ext3 FAQ:

Q: How can I recover (undelete) deleted files from my ext3 partition?
Actually, you can’t! This is what one of the developers, Andreas Dilger, said about it:

In order to ensure that ext3 can safely resume an unlink after a crash, it actually zeros out the block pointers in the inode, whereas ext2 just marks these blocks as unused in the block bitmaps and marks the inode as “deleted” and leaves the block pointers alone.
Your only hope is to “grep” for parts of your files that have been deleted and hope for the best.

The process, as described in this article, is the “grep” that Andreas is referring to. Hopefully, as ext3 is developed further, some effort will be put in to making this process easier and more reliable.

Conclusion

While going through this process may be necessary to recover information lost in any number of situations, it’s not a process you want to go through on a Monday morning to recover your organization’s payroll data after an administrator fat-fingers an rm command. The single most important piece of information you should take away from this article, in that vein, is to keep current, tested backups of business critical data that reside on the systems you manage. Regardless of the reason for its use, the process covered in this article is something that every system administrator and security analyst should have in their toolbelt.

Source

APC is the Alternative PHP Cache, which is a free, open, and robust framework for caching and optimizing PHP intermediate code. What this means is that APC reads your PHP files, parses them into a more efficient binary format and then caches them in memory so that each request for your PHP files and PHP library files can be fed from the parsed cache. This will generally lead to a speed increase when serving a PHP site, especially one with a lot of library files. This post looks at how to install APC for PHP on Linux. The Linux distribution I used was CentOS 5, but it should be fairly similar for most distros.

First of all you need to download the APC code from the PHP PECL library.  Change directory to somewhere like /usr/local/src and then get the latest version:

$ wget http://pecl.php.net/get/APC

–22:58:41–  http://pecl.php.net/get/APC
Resolving pecl.php.net… 216.92.131.66
Connecting to pecl.php.net|216.92.131.66|:80… connected.
HTTP request sent, awaiting response… 200 OK
Length: 114298 (112K) [application/octet-stream]
Saving to: `APC-3.0.16.tgz’

100%[=====================>] 114,298     97.1K/s   in 1.1s

22:58:43 (97.1 KB/s) – `APC-3.0.16.tgz’ saved [114298/114298]

So then you need to extract the files:

$ tar -zxf APC-3.0.16.tgz

and change into the APC directory:

$ cd APC-3.0.16

The next step is to run the “phpize” command. This requires that you have PHP development package installed. On CentOS this is php-devel (installed by running “yum install php-devel”) and it should have a similar name on other Linux distros.

$ phpize

Configuring for:
PHP Api Version:         20041225
Zend Module Api No:      20050922
Zend Extension Api No:   220051025

You then configure APC, telling it where the executable file php-config is. If you don’t know where this is, then do this:

$ whereis php-config

which will return something like:

php-config: /usr/bin/php-config /usr/share/man/man1/php-config.1.gz

and then run the configure command like so:

./configure –enable-apc –enable-apc-mmap –with-apxs –with-php-config=/usr/bin/php-config

This will go ahead and do some configuring stuff which will look something like this:

checking for egrep… grep -E
checking for a sed that does not truncate output… /bin/sed
…
checking dynamic linker characteristics… GNU/Linux ld.so
checking how to hardcode library paths into programs… immediate
configure: creating ./config.status
config.status: creating config.h

Now that configure is done, it’s just a matter of running make :

$ make

…
Libraries have been installed in:
/tmp/APC-3.0.16/modules

If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR’
flag during linking and do at least one of the following:
- add LIBDIR to the `LD_LIBRARY_PATH’ environment variable
during execution
- add LIBDIR to the `LD_RUN_PATH’ environment variable
during linking
- use the `-Wl,–rpath -Wl,LIBDIR’ linker flag
- have your system administrator add LIBDIR to `/etc/ld.so.conf’

See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
———————————————————————-

Build complete.
(It is safe to ignore warnings about tempnam and tmpnam).

and then make install:

$ make install

Installing shared extensions:     /usr/lib64/php/modules/

and restarting the Apache web server:

/etc/init.d/httpd restart

The APC cache will now be enabled. You can confirm this by creating a script which calls the phpinfo() command and looking for the APC section. It will have been switched on by default by adding a “extension=apc.so” line to your /etc/php.ini file, but you may want to add more settings to configure it more.

The INSTALL file suggests this (I have put the default values at the end of each line which is what is set if you don’t set anything in the php.ini file):

apc.enabled=1                       # default = 1
apc.shm_segments=1                  # default = 1
apc.shm_size=128                    # default = 30
apc.ttl=7200                        # default = 0
apc.user_ttl=7200                   # default = 0
apc.num_files_hint=1024             # default = 1000
apc.mmap_file_mask=/tmp/apc.XXXXXX  # default = no value
apc.enable_cli=1                    # default = 0

And that’s all there is to it. There is also a monitoring script available so you can see what’s being cached and how much memory is being used etc. You can read about this in my “Displaying PHP APC Cache Information” post.

Update November 5th 2008: I just installed this on another server and I didn’t have any issues installing it but it didn’t add the extension=apc.so line to the /etc/php.ini file automatically. So if it doesn’t appear to be working, check the php.ini to ensure the line is there and add it if not.

Source

Step 1: Back up important files

Red Hat has finally placed sendmail.cf in /etc/mail, where it belongs. To verify the location of your configuration file, type this command:

sendmail -d0.20 -bv | grep sendmail.cf

The default installation outputs this:

Conf file: /etc/mail/sendmail.cf (default for MTA)
Conf file: /etc/mail/sendmail.cf (selected)

Be sure to use this path when generating your new sendmail.cf from sendmail.mc, or no changes will take place. Back up your current sendmail.cf and the m4 file that generated it (probably /etc/mail/sendmail.mc):

cp /etc/mail/sendmail.cf /etc/mail/sendmail.cf~
cp /etc/mail/sendmail.mc /etc/mail/sendmail.mc~

Step 2: Make your certificate

We are also setting up STARTTLS, which allows sendmail to communicate over an encrypted layer using TLS. This is very important, as it allows us to use the LOGIN or PLAIN authentication mechanisms without transferring the password in plain text. It also allows the entire message to remain encrypted from the user’s machine to the mail server. If sendmail relays the message to another server that offers STARTTLS, the message will be encrypted again. But the most important advantage of this approach is that we get to authenticate using regular system logins and passwords, with no need to maintain a separate user database.

Red Hat’s openssl package includes a Makefile that makes it extremely easy to generate a certificate (note that on Fedora Core 4 the location is now /etc/pki/tls/certs):

cd /usr/share/ssl/certs
make sendmail.pem

Just follow the prompts and be sure to use the fully qualified domain name of the mail server for the Common Name prompt. Users will still be warned that the certificate is self-signed or not trusted, but you will prevent a warning that the certificate doesn’t match the host offering it. This certificate is suitable for testing, but you may want to investigate further about the use of certificates before deploying it in a production environment, a topic that is beyond the scope of this howto.

Step 3: Edit sendmail.mc

If you take a look at the sendmail.mc provided by Red Hat, you will notice that the necessary directives are already present but have been commented out (m4 doesn’t use the # symbol for comments, it starts a line with dnl, which stands for “delete until new line”). Since we want the easiest method possible, without sacrificing security, we need to edit these lines. Don’t cut & paste from this web page, or you may introduce unwanted characters into your configuration file that will prevent sendmail from starting.

The confAUTH_OPTIONS macro allows you to instruct sendmail not to offer plain text authentication until after a secure mechanism such as TLS is active (the p option). We are also prohibiting anonymous logins (the y option). The A option is a workaround for broken MTAs:

define(`confAUTH_OPTIONS’, `A p y’)dnl

Now we define which authentication mechanisms we will trust and use:

TRUST_AUTH_MECH(`LOGIN PLAIN’)dnl
define(`confAUTH_MECHANISMS’, `LOGIN PLAIN’)dnl

Next, we tell sendmail where to find the certificates:

define(`confCACERT_PATH’,`/usr/share/ssl/certs’)
define(`confCACERT’,`/usr/share/ssl/certs/ca-bundle.crt’)
define(`confSERVER_CERT’,`/usr/share/ssl/certs/sendmail.pem’)
define(`confSERVER_KEY’,`/usr/share/ssl/certs/sendmail.pem’)

And finally, it may be useful to increase the log level for debugging purposes (delete or comment out this line after everything is working properly):

define(`confLOG_LEVEL’, `14′)dnl

Use the m4 command to generate a new sendmail.cf:

m4 /etc/mail/sendmail.mc > /etc/mail/sendmail.cf

Be sure to use the right location for sendmail.cf, as determined earlier. Alternatively, you can use the following command in a stock Red Hat 9.0 or Fedora Core installation:

make -C /etc/mail sendmail.cf

This uses the commands in /etc/mail/Makefile to generate the new sendmail.cf configuration file.

Step 4: Test the configuration

This is where things get really interesting. sendmail must be restarted before it can use the new configuration file. Rather than simply restarting sendmail with our fingers crossed, we can test it to verify that every thing works properly. You can stop sendmail and then start it with command line options that cause it to log to a specified file. There are various ways to stop sendmail on a Red Hat/Fedora system:

service sendmail stop

or

cd /etc/mail
make stop

or

make -C /etc/mail stop

or

/etc/init.d/sendmail stop

We want to start sendmail with arguments to make it log the SMTP transaction to a special file while we are testing it:

sendmail -bD -X /tmp/test.log

Now, try to send a message from an e-mail client on another computer that does not have relay access, using your server as the outgoing mail server. You should be denied relaying. Edit your preferences so that the client uses authentication, with a login and password (not Secure Password Authentication, or SPA, which is something completely different). You should still be denied access. The last thing you need to do is to instruct the client to use SSL or TLS with the outgoing mail server (there is no need to specify a special port). After making this change, you should be able to send mail (you will be prompted to accept the certificate, however, which you might want to install to prevent further prompts). Now hit ctrl-c to stop sendmail. Restart it normally:

service sendmail restart

Now it’s time to look at the log. After the first EHLO, sendmail offers something like this:

30245 >>> 250-ENHANCEDSTATUSCODES
30245 >>> 250-PIPELINING
30245 >>> 250-8BITMIME
30245 >>> 250-SIZE
30245 >>> 250-DSN
30245 >>> 250-ETRN
30245 >>> 250-STARTTLS
30245 >>> 250-DELIVERBY
30245 >>> 250 HELP

The important thing is that AUTH is not offered here, because the channel isn’t encrypted. If you see AUTH in the first exchange, and it offers PLAIN or LOGIN, something is wrong. Look at your logs, go over the previous steps, and make sure that you generated a new sendmail.cf in the right location. The next entries in our log show that TLS is activated:

30245 <<< STARTTLS
30245 >>> 220 2.0.0 Ready to start TLS

Another EHLO takes place, followed by something like this:

30245 >>> 250-ENHANCEDSTATUSCODES
30245 >>> 250-PIPELINING
30245 >>> 250-8BITMIME
30245 >>> 250-SIZE
30245 >>> 250-DSN
30245 >>> 250-ETRN
30245 >>> 250-AUTH LOGIN PLAIN
30245 >>> 250-DELIVERBY
30245 >>> 250 HELP

Now AUTH is offered with the allowed mechanisms (but not STARTTLS, which isn’t needed here, as the channel is already encrypted). Authentication takes place, and the message is relayed to its destination.

It’s interesting to note that the username and password is Base64 encoded by the client, so it isn’t really sent as clear text:

30245 <<< AUTH PLAIN AHJvYmVydABzbHVncw==
30245 >>> 235 2.0.0 OK Authenticated

Nevertheless, it would be trivial to decode the string into the correct username/login pair (robert/slugs, in this case). Therefore, it is best to secure the transaction with TLS. If you want to verify that the transaction is encrypted, open another terminal for root, and run tcpdump:

tcpdump -s 1500 -vvxX port 25

Send a mail with easy to identify strings. You shouldn’t see your login or the message in tcpdump’s output.

Note that the certificate will be exchanged in plain text before TLS is enabled. If the mail is relayed to another server that doesn’t offer STARTTLS, you will see the content of the outgoing message in plain text.

Source

Just do as followings. Login root or as correct owner for the folder:

# cd (to your /path/directory)
# find -type d -print0 |xargs -0 chmod 755
# find -type f -print0 |xargs -0 chmod 644

Cleaning the Mail Spool

There could be a lot of “trash” qf or df files left behind following a bout of misbehaving sendmails. If the size of either file is 0, you should be able to trash them safely.

cd /var/spool/mqueue-fixme

to get into your queue directory, and

find . -size 0 -exec rm {} \;

to find everything with a size of 0 and execute the remove command on the found files. This is unwise to run in your default queue directory, even when sendmail is stopped.

Perhaps a sleazy spammer, cretinous chain-mailer, naughty Novell server, or administrator accidentally sending out e-mail to everyone is the cause of your mail woes. You can pull these messages from the mail queue thusly:

cd /var/spool/mqueue-fixme
mkdir /var/tmp/EVIDENCE-OF-ILLDOING
grep idiot@wherever qf* | cut -d":" -f1  | uniq | cut -d"f" -f2 | \
 xargs -i echo "mv *{} /var/tmp/EVIDENCE-OF-ILLDOING" > RUNME
chmod 700 RUNME
./RUNME

A sendmail problem could result in lots of “qf” files with no corresponding “df” file, and vice versa. You can get rid of these unmatched files with:

cd /var/spool/mqueue-fixme
ls -1 | cut -c 3-16 | sort | uniq -c | sort -n | grep " 1 " | awk '{print "*"$2}' > ~/rmfile

Then, execute this perl script:

#!/usr/local/bin/perl

use strict;

open (RMFILE, "rmfile");
while (my $line=) {
        chomp $line;
        print $line."\n";
        system ("rm /var/spool/mqueue/$line");
}
close (RMFILE);

I realize that this is not pretty, but it was the best that I could come up with since echo and xargs were not cooperating with me whatsoever.

Source

Originally by Scott Hebert

* his article has been edited to fit in my case.

JSON (JavaScript Object Notation) LogoI recently started messing around with building my own Facebook application. I figured the best way to start learning was to download the demo application and get it working on the Slaptijack web server. I quickly ran into a problem:

PHP Fatal error:  Call to undefined function json_encode()

The problem is that PHP 5.2 includes the json_encode() and json_decode() functions built right in. Unfortunately, I’m running on PHP 5.0 which doesn’t include native JSON support. Here’s how I resolved that.

1. Install json – This was actually trickier than expected. I assumed I would be able to install this via pear. Apparently, a PEAR Services_JSON package was developed, but it has never been accepted into the official repository.

The trick instead is to use the PECL json package. This was as easy as running pecl install json and watch the compiler do its thing.

When it’s done you should have a json.so file in your PHP modules directory. Usually under: /usr/lib/php/modules/

2. Create a file with nano or vi, named : json.ini and put it in to /etc/php.d/ folder – and simply add extension=json.so to this file and that will enable the extension. *skip this step if you don’t have /etc/php.d/ folder*

In other case, you can just need to add: extension=json.so inside your php.ini file. Easy way, you can run below command from your root access :

echo “extension=json.so” >> [directory_path_to]/php.ini

3. Restart Apache - Not much more to add here. Without the restart, the extension won’t be loaded.

4. Profit!

That’s all it took. Now my PHP 5 installation is kicking along happily with the required JSON functions.

Original source:
http://slaptijack.com/system-administration/lets-install-json-for-php-5/

What you want is possible but it is considerably more work that it is practical to put in. Just say decode and let PHP do the calculations

Anyway, thanks for an interesting question. Researching it taught me about both how UTF-8 works and about URL encoding in general.

First, link to an explanation of URL encoding:
http://www.blooberry.com/indexdot/html/topics/urlencoding.htm
(disclosure: it’s written by someone I know

Secondly, here is how to find the character from a URL encoding – manually!

Your character above – “我” (according to babelfish.altavista.com it means “I” in Chinese, if you can’t see it in your browser try to copy this and paste in your address bar: javascript:’<html>我