Source:
http://www.ducea.com/2008/08/19/sendmail-multiple-queues/

Sendmail will use by default a single mail queue. This is what most users will need, and if you don’t have any special requirement you will not care about this. Still for high traffic mail servers it might be useful to split the queue over several directories, as thousands of files in a single directory will become a performance penalty at some point and also processing the queue sequentially will become very slow.

This post will show how we can implement multiple mail queues with modern sendmail versions.
Let’s start by assuming we want to use 8 mail queues. First thing is to create the actual directories as sendmail will not do this by default:

mkdir /var/spool/mqueue/q{1,2,3,4,5,6,7,8}

And fix the permissions to the ones of the original folder /var/spool/mqueue. For ex. this might look like:

chown -R root:smmsp /var/spool/mqueue/q*

using a default sendmail install running on debian. Fix the users to the specific ones found on your system (ls -al /var/spool/mqueue if you are uncertain of this).

Next, we need to enable the multiple queues in the sendmail configuration. For this we will edit sendmail.mc (normally found under /etc/mail) and append one line:

define(`QUEUE_DIR', `/var/spool/mqueue/q*')dnl

and now regenerate sendmail.cf; this is done normally running:

m4 sendmail.mc > /etc/mail/sendmail.cf

(fix your paths appropriately), or if you are using debian sendmail you can just run make all in /etc/mail.

After restarting sendmail, it will start using the multiple queues we defined. Running mailq will output each of the queues:

#mailq
/var/spool/mqueue/q6 is empty
/var/spool/mqueue/q4 is empty
/var/spool/mqueue/q3 is empty
/var/spool/mqueue/q2 is empty
/var/spool/mqueue/q5 is empty
/var/spool/mqueue/q1 is empty
/var/spool/mqueue/q7 is empty
/var/spool/mqueue/q8 is empty
Total requests: 0

Note: if you want to add more folders to the configuration all you have to do is to create the respective folders, set the appropriate permissions and restart sendmail.

If you had existing mails in the queue (most likely if you were looking for this solution), if you want them still processed, move them from /var/spool/mqueue in one of the newly created queues (q1 for ex).

Individual queue directories can be symbolic links to other partitions to spreads load among multiple disks. Queue IDs are unique across queues so you can move the items among queues if you have to.

This is what I’m currently using on “sendmail.mc”. So far quite good and I can blast around 100K emails within few hours. Enjoy!

divert(-1)dnl
include(`/usr/share/sendmail-cf/m4/cf.m4′)dnl
VERSIONID(`setup for linux’)dnl
OSTYPE(`linux’)dnl
define(`confDEF_USER_ID’, “8:12”)dnl
dnl define(`confAUTO_REBUILD’)dnl
define(`confTO_CONNECT’, `1m’)dnl
define(`confTRY_NULL_MX_LIST’, `True’)dnl
define(`confDONT_PROBE_INTERFACES’, `True’)dnl
define(`PROCMAIL_MAILER_PATH’, `/usr/bin/procmail’)dnl
define(`ALIAS_FILE’, `/etc/aliases’)dnl
define(`STATUS_FILE’, `/var/log/mail/statistics’)dnl
define(`UUCP_MAILER_MAX’, `2000000′)dnl
define(`confUSERDB_SPEC’, `/etc/mail/userdb.db’)dnl
define(`confPRIVACY_FLAGS’, `authwarnings,novrfy,noexpn,restrictqrun’)dnl
define(`confAUTH_OPTIONS’, `A’)dnl
define(`confCHECKPOINTINTERVAL’,`0′)dnl
define(`confCONNECTION_RATE_THROTTLE’,`0′)dnl
define(`confDF_BUFFER_SIZE’,`16384′)dnl
define(`confMAX_DAEMON_CHILDREN’,`0′)dnl
define(`confMAX_QUEUE_RUN_SIZE’,`0′)dnl
define(`confMCI_CACHE_SIZE’,`4′)dnl
define(`confMCI_CACHE_TIMEOUT’,`120s’)dnl
define(`confMIN_QUEUE_AGE’,`0′)dnl
define(`confSAFE_QUEUE’,`false’)dnl
define(`confTO_IDENT’,`0′)dnl
define(`confXF_BUFFER_SIZE’,`16384′)dnl
define(`confQUEUE_LA’,`1000′)dnl
define(`confREFUSE_LA’,`500′)dnl
FEATURE(`nocanonify’, `canonify_hosts’)dnl
FEATURE(`no_default_msa’, `dnl’)dnl
FEATURE(`mailertable’, `hash -o /etc/mail/mailertable.db’)dnl
FEATURE(`virtusertable’, `hash -o /etc/mail/virtusertable.db’)dnl
FEATURE(redirect)dnl
dnl # FEATURE(always_add_domain)dnl
FEATURE(use_cw_file)dnl
FEATURE(use_ct_file)dnl
FEATURE(local_procmail, `’, `procmail -t -Y -a $h -d $u’)dnl
FEATURE(`access_db’, `hash -T<TMPF> -o /etc/mail/access.db’)dnl
EXPOSED_USER(`root’)dnl
DAEMON_OPTIONS(`Name=MTA-v4, Family=inet’)
LOCAL_DOMAIN(`localhost.localdomain’)dnl
MODIFY_MAILER_FLAGS(`PROCMAIL’, `+m’)dnl
dnl # INPUT_MAIL_FILTER(`dk-filter’, `S=inet:8891@localhost’)dnl
FEATURE(`dnsbl’,`bl.spamcop.net’,`554 Mail from $&{client_addr} rejected by bl.spamcop.net’)dnl
FEATURE(`dnsbl’,`rbl-plus.mail-abuse.org’,`”MAPS-listed host: http://mail-abuse.org/cgi-bin/lookup?”$&{client_addr}’)dnl
FEATURE(`dnsbl’,`sbl-xbl.spamhaus.org’,`554 Mail from $&{client_addr} has been rejected by the Spamhaus Blackhole List’)dnl
FEATURE(`dnsbl’,`dnsbl.sorbs.net’,`554 Mail from $&{client_addr} has been rejected by the SORBS’)dnl
MAILER(smtp)dnl
MAILER(procmail)dnl

processor : 1
vendor_id : AuthenticAMD
cpu family : 16
model : 2
model name : Quad-Core AMD Opteron(tm) Processor 2350 HE
stepping : 3
cpu MHz : 1995.000
cache size : 512 KB

processor : 2
vendor_id : AuthenticAMD
cpu family : 16
model : 2
model name : Quad-Core AMD Opteron(tm) Processor 2350 HE
stepping : 3
cpu MHz : 1995.000
cache size : 512 KB

processor : 3
vendor_id : AuthenticAMD
cpu family : 16
model : 2
model name : Quad-Core AMD Opteron(tm) Processor 2350 HE
stepping : 3
cpu MHz : 1995.000
cache size : 512 KB

# Copyright (c) 2001 Sendmail, Inc. and its suppliers.
# All rights reserved.
#
# By using this file, you agree to the terms and conditions set
# forth in the LICENSE file which can be found at the top level of
# the sendmail distribution.
#
# $Id: TUNING,v 1.16 2001/08/19 21:03:38 gshapiro Exp $
#

********************************************
** This is a DRAFT, comments are welcome! **
********************************************

If the default configuration of sendmail does not achieve the
required performance, there are several configuration options that
can be changed to accomplish higher performance. However, before
those options are changed it is necessary to understand why the
performance is not as good as desired. This may also involve hardware
and software (OS) configurations which are not extensively explored
in this document. We assume that your system is not limited by
network bandwidth because optimizing for this situation is beyond
the scope of this guide. In almost all other cases performance will
be limited by disk I/O.

This text assumes that all options which are mentioned here are
familiar to the reader, they are explained in the Sendmail Installation
and Operations Guide; doc/op/op.txt.

There are basically three different scenarios which are treated
in the following:
* Mailing Lists and Large Aliases (1-n Mailing)
* 1-1 Mass Mailing
* High Volume Mail

Depending on your requirements, these may need different options
to optimize sendmail for the particular purpose. It is also possible
to configure sendmail to achieve good performance in all cases, but
it will not be optimal for any specific purpose. For example, it
is non-trivival to combine low latency (fast delivery of incoming
mail) with high overall throughput.

Before we explore the different scenarios, a basic discussion about
disk I/O, delivery modes, and queue control is required.

* Disk I/O
———————————————–

In general mail will be written to disk up before a delivery attempt
is made. This is required for reliability and should only be changed
in a few specific cases that are mentioned later on. To achieve
better disk I/O performance the queue directories can be spread
over several disks to distribute the load. This is some basic tuning
that should be done in all cases where the I/O speed of a single
disk is exceeded, which is true for almost every high-volume
situation except if a special disk subsystem with large (NV)RAM
buffer is used.

Depending on your OS there might be ways to speed up I/O, e.g.,
using softupdates or turning on the noatime mount option. If this
is done make sure the filesystem is still reliable, i.e., if fsync()
returns without an error, the file has really been committed to
disk.

* Queueing Strategies and DeliveryMode
———————————————–

There are basically three delivery modes:

background: incoming mail will be immediately delivered by a new process
interactive: incoming mail will be immediately delivered by the same process
queue: incoming mail will be queued and delivered by a queue runner later on

The first offers the lowest latency without the disadvantage of the
second, which keep the connection from the sender open until the
delivery to the next hop succeeded or failed. However, it does not
allow for a good control over the number of delivery processes other
than limiting the total number of direct children of the daemon
processes (MaxChildren) or by load control options (RefuseLA,
DelayLA). Moreover, it can’t make as good use as ‘queue’ mode can
for connection caching.

Interactive DeliveryMode should only be used in rare cases, e.g.,
if the delivery time to the next hop is a known quantity or if the
sender is under local control and it does not matter if it has to
wait for delivery.

Queueing up e-mail before delivery is done by a queue runner allows
the best load control but does not achieve as low latency as the
other two modes. However, this mode is probably also best for
concurrent delivery since the number of queue runners can be specified
on a queue group basis. Persistent queue runners (-qp) can be used
to minimize the overhead for creating processes because they just
sleep for the specified interval (which shold be short) instead of
exiting after a queue run.

* Queue Groups
———————————————–

In most situations disk I/O is a bottleneck which can be mitigated
by spreading the load over several disks. This can easily be achieved
with different queue directories. sendmail 8.12 introduces queue
groups which are collections of queue directories with similar
properties, i.e., number of processes to run the queues in the
group, maximum number of recipients within an e-mail (envelope),
etc. Queue groups allow control over the behaviour of different
queues. Depending on the setup, it is usually possible to have
several queue runners delivering mails concurrently which should
increase throughput. The number of queue runners can be controlled
per queue group (Runner=) and overall (MaxQueueChildren).

* DNS Lookups
———————————————–

sendmail performs by default host name canonifications by using
host name lookups. This process is meant to replace unqualified
host name with qualified host names, and CNAMEs with the non-aliased
name. However, these lookups can take a while for large address
lists, e.g., mailing lists. If you can assure by other means that
host names are canonical, you should use

FEATURE(`nocanonify’, `canonify_hosts’)

in your .mc file. For further information on this feature and
additional options see cf/README. If sendmail is invoked directly
to send e-mail then either the -G option should be used or

define(`confDIRECT_SUBMISSION_MODIFIERS’, `C’)

should be added to the .mc file.

* Mailing Lists and Large Aliases (1-n Mailing)
———————————————–

Before 8.12 sendmail delivers an e-mail sequentially to all its
recipients. For mailing lists or large aliases the overall delivery
time can be substantial, especially if some of the recipients are located
at hosts that are slow to accept e-mail. Some mailing list software
therefore “split” up e-mails into smaller pieces with fewer recipients.
sendmail 8.12 can do this itself, either across queue groups or
within a queue directory. For the former the option SplitAcrossQueueGroups
option must be set, the latter is controlled by the ‘r=’ field of
a queue group declaration.

Let’s assume a simple example: a mailing lists where most of
the recipients are at three domains: the local one (local.domain)
and two remotes (one.domain, two.domain) and the rest is splittered
over several other domains. For this case it is useful to specify
three queue groups:

QUEUE_GROUP(`local’, `P=/var/spool/mqueue/local, F=f, R=2, I=1m’)dnl
QUEUE_GROUP(`one’, `P=/var/spool/mqueue/one, F=f, r=50, R=3′)dnl
QUEUE_GROUP(`two’, `P=/var/spool/mqueue/two, F=f, r=30, R=4′)dnl
QUEUE_GROUP(`remote’, `P=/var/spool/mqueue/remote, F=f, r=5, R=8, I=2m’)dnl
define(`ESMTP_MAILER_QGRP’, `remote’)dnl
define(`confSPLIT_ACROSS_QUEUEGROUPS’, `True’)dnl
define(`confDELIVERY_MODE’, `q’)dnl
define(`confMAX_QUEUE_CHILDREN’, `50′)dnl
define(`confMIN_QUEUE_AGE’, `27m’)dnl

and specify the queuegroup ruleset as follows:

LOCAL_RULESETS
Squeuegroup
R$* @ local.domain $# local
R$* @ $* one.domain $# one
R$* @ $* two.domain $# two
R$* @ $* $# remote
R$* $# mqueue

Now it is necessary to control the number of queue runners, which
is done by MaxQueueChildren. Starting the daemon with the option
-q5m assures that the first delivery attempt for each e-mail is
done within 5 minutes, however, there are also individual queue
intervals for the queue groups as specified above. MinQueueAge
is set to 27 minutes to avoid that entries are run too often.

Notice: if envelope splitting happens due to alias expansion, and
DeliveryMode is not ‘i’nteractive, then only one envelope is sent
immediately. The rest (after splitting) are queued up and queue
runners must come along and take care of them. Hence it is essential
that the queue interval is very short.

* 1-1 Mass Mailing
———————————————–

In this case some program generates e-mails which are sent to
individual recipients (or at most very few per e-mail). A simple
way to achieve high throughput is to set the delivery mode to
‘interactive’, turn off the SuperSafe option and make sure that the
program that generates the mails can deal with mail losses if the
server loses power. In no other case should SuperSafe be set to
‘false’. If these conditions are met, sendmail does not need to
commit mails to disk but can buffer them in memory which will greatly
enhance performance, especially compared to normal disk subsystems, e.g.,
non solid-state disks.

* High Volume Mail
———————————————–

For high volume mail it is necessary to be able to control the load
on the system. Therefore the ‘queue’ delivery mode should be used,
and all options related to number of processes and the load should
be set to reasonable values. It is important not to accept mail
faster than it can be delivered otherwise the system will be
overwhelmed. Hence RefuseLA should be lower than QueueLA, the number
of daemon children should probably be lower than the number of queue
runnners (MaxChildren vs. MaxQueueChildren). DelayLA is a new option
in 8.12 which allows delaying connections instead of rejecting them.
This may result in a smoother load distribution depending on how
the mails are submitted to sendmail.

* Miscellaneous
———————————————–

Other options that are interesting to tweak performance are
(in no particular order):

SuperSafe: if interactive DeliveryMode is used, then this can
be set to the new value “interactive” in 8.12 to save some disk
synchronizations which are not really necessary in that mode.

———————————————–
Source:
http://luxio.us/gXwyLu

Source: Stupid HTAccess Tricks

GENERAL INFORMATION [ ^ ]

.htaccess Definition ¹ ^

Apache server software provides distributed (i.e., directory-level) configuration via Hypertext Access files. These .htaccess files enable the localized fine-tuning of Apache’s universal system-configuration directives, which are defined in Apache’s main configuration file. The localized .htaccess directives must operate from within a file named .htaccess. The user must have appropriate file permissions to access and/or edit the .htaccess file. Further,.htaccess file permissions should never allow world write access — a secure permissions setting is “644”, which allows universal read access and user-only write access. Finally,.htaccess rules apply to the parent directory and all subdirectories. Thus to apply configuration rules to an entire website, place the .htaccess file in the root directory of the site.

Commenting .htaccess Code ^

Comments are essential to maintaining control over any involved portion of code. Comments in.htaccess code are fashioned on a per-line basis, with each line of comments beginning with a pound sign #. Thus, comments spanning multiple lines in the .htaccess file require multiple pound signs. Further, due to the extremely volatile nature of htaccess voodoo, it is wise to include only alphanumeric characters (and perhaps a few dashes and underscores) in any.htaccess comments.

Important Notes for .htaccess Noobs ^

As a configuration file, .htaccess is very powerful. Even the slightest syntax error (like a missing space) can result in severe server malfunction. Thus it is crucial to make backup copies of everything related to your site (including any original .htaccess files) before working with your Hypertext Access file(s). It is also important to check your entire website thoroughly after making any changes to your .htaccess file. If any errors or other problems are encountered, employ your backups immediately to restore original functionality.

Performance Issues ^

.htaccess directives provide directory-level configuration without requiring access to Apache’s main server cofiguration file (httpd.conf). However, due to performance and security concerns, the main configuration file should always be used for server directives whenever possible. For example, when a server is configured to process .htaccess directives, Apache must search every directory within the domain and load any and all .htaccess files upon every document request. This results in increased page processing time and thus decreases performance. Such a performance hit may be unnoticeable for sites with light traffic, but becomes a more serious issue for more popular websites. Therefore, .htaccess files should only be used when the main server configuration file is inaccessible. See the “Performance Tricks” section of this article for more information.

Regex Character Definitions for htaccess ² ^

#

the # instructs the server to ignore the line. used for including comments. each line of comments requires it’s own #. when including comments, it is good practice to use only letters, numbers, dashes, and underscores. this practice will help eliminate/avoid potential server parsing errors.

[F]

Forbidden: instructs the server to return a 403 Forbidden to the client.

[L]

Last rule: instructs the server to stop rewriting after the preceding directive is processed.

[N]

Next: instructs Apache to rerun the rewrite rule until all rewriting directives have been achieved.

[G]

Gone: instructs the server to deliver Gone (no longer exists) status message.

[P]

Proxy: instructs server to handle requests by mod_proxy

[C]

Chain: instructs server to chain the current rule with the previous rule.

[R]

Redirect: instructs Apache to issue a redirect, causing the browser to request the rewritten/modified URL.

[NC]

No Case: defines any associated argument as case-insensitive. i.e., “NC” = “No Case”.

[PT]

Pass Through: instructs mod_rewrite to pass the rewritten URL back to Apache for further processing.

[OR]

Or: specifies a logical “or” that ties two expressions together such that either one proving true will cause the associated rule to be applied.

[NE]

No Escape: instructs the server to parse output without escaping characters.

[NS]

No Subrequest: instructs the server to skip the directive if internal sub-request.

[QSA]

Append Query String: directs server to add the query string to the end of the expression (URL).

[S=x]

Skip: instructs the server to skip the next “x” number of rules if a match is detected.

[E=variable:value]

Environmental Variable: instructs the server to set the environmental variable “variable” to “value”.

[T=MIME-type]

Mime Type: declares the mime type of the target resource.

[]

specifies a character class, in which any character within the brackets will be a match. e.g., [xyz] will match either an x, y, or z.

[]+

character class in which any combination of items within the brackets will be a match. e.g., [xyz]+ will match any number of x’s, y’s, z’s, or any combination of these characters.

[^]

specifies not within a character class. e.g., [^xyz] will match any character that is neither x, y, nor z.

[a-z]

a dash (-) between two characters within a character class ([]) denotes the range of characters between them. e.g., [a-zA-Z] matches all lowercase and uppercase letters from a to z.

a{n}

specifies an exact number, n, of the preceding character. e.g., x{3} matches exactly threex’s.

a{n,}

specifies n or more of the preceding character. e.g., x{3,} matches three or more x’s.

a{n,m}

specifies a range of numbers, between n and m, of the preceding character. e.g., x{3,7} matches three, four, five, six, or seven x’s.

()

used to group characters together, thereby considering them as a single unit. e.g., (perishable)?press will match press, with or without the perishable prefix.

^

denotes the beginning of a regex (regex = regular expression) test string. i.e., begin argument with the proceeding character.

$

denotes the end of a regex (regex = regular expression) test string. i.e., end argument with the previous character.

?

declares as optional the preceding character. e.g., monzas? will match monza or monzas, while mon(za)? will match either mon or monza. i.e., x? matches zero or one of x.

!

declares negation. e.g., “!string” matches everything except “string”.

.

a dot (or period) indicates any single arbitrary character.

-

instructs “not to” rewrite the URL, as in “...domain.com.* - [F]”.

+

matches one or more of the preceding character. e.g., G+ matches one or more G’s, while “+” will match one or more characters of any kind.

*

matches zero or more of the preceding character. e.g., use “.*” as a wildcard.

|

declares a logical “or” operator. for example, (x|y) matches x or y.

\

escapes special characters ( ^ $ ! . * | ). e.g., use “\.” to indicate/escape a literal dot.

\.

indicates a literal dot (escaped).

/*

zero or more slashes.

.*

zero or more arbitrary characters.

^$

defines an empty string.

^.*$

the standard pattern for matching everything.

[^/.]

defines one character that is neither a slash nor a dot.

[^/.]+

defines any number of characters which contains neither slash nor dot.

http://

this is a literal statement — in this case, the literal character string, “http://”.

^domain.*

defines a string that begins with the term “domain”, which then may be proceeded by any number of any characters.

^domain\.com$

defines the exact string “domain.com”.

-d

tests if string is an existing directory

-f

tests if string is an existing file

-s

tests if file in test string has a non-zero value

Redirection Header Codes ^

• 301 – Moved Permanently
• 302 – Moved Temporarily
• 403 – Forbidden
• 404 – Not Found
• 410 – Gone

ESSENTIALS [ ^ ]

Commenting your htaccess Files ^

It is an excellent idea to consistenly and logically comment your htaccess files. Any line in an htaccess file that begins with the pound sign ( # ) tells the server to ignore it. Multiple lines require multiple pounds and use letters/numbers/dash/underscore only:

# this is a comment
# each line must have its own pound sign
# use only alphanumeric characters along with dashes - and underscores _

Enable Basic Rewriting ^

Certain servers may not have “mod_rewrite” enabled by default. To ensure mod_rewrite(basic rewriting) is enabled throughout your site, add the following line once to your site’s root htaccess file:

# enable basic rewriting
RewriteEngine on

Enable Symbolic Links ^

Enable symbolic links (symlinks) by adding the following directive to the target directory’s htaccess file. Note: for the FollowSymLinks directive to function, AllowOverride Optionsprivileges must be enabled from within the server configuration file (see proceeding paragraph for more information):

# enable symbolic links
Options +FollowSymLinks

Enable AllowOverride ^

For directives that require AllowOverride in order to function, such as FollowSymLinks (see above paragraph), the following directive must be added to the server configuration file. For performance considerations, it is important to only enable AllowOverride in the specific directory or directories in which it is required. In the following code chunk, we are enabling theAllowOverride privs only in the specified directory (/www/replace/this/with/actual/directory). Refer to this section for more information about AllowOverride and performance enhancement:

# enable allowoverride privileges
<Directory /www/replace/this/with/actual/directory>
AllowOverride Options
</Directory>

Rename the htaccess File ^

Not every system enjoys the extension-only format of htaccess files. Fortunately, you can rename them to whatever you wish, granted the name is valid on your system. Note: This directive must be placed in the server-wide configuration file or it will not work:

# rename htaccess files
AccessFileName ht.access

Note: If you rename your htaccess files, remember to update any associated configuration settings. For example, if you are protecting your htaccess file via FilesMatch, remember to inform it of the renamed files:

# protect renamed htaccess files
<FilesMatch "^ht\.">
Order deny,allow
Deny from all
</FilesMatch>

Retain Rules Defined in httpd.conf ^

Save yourself time and effort by defining replicate rules for multiple virtual hosts once and only once via your httpd.conf file. Then, simply instruct your target htaccess file(s) to inherit the httpd.conf rules by including this directive:

RewriteOptions Inherit

PERFORMANCE [ ^ ]

Improving Performance via AllowOverride ^

Limit the extent to which htaccess files decrease performance by enabling AllowOverride only in required directories. For example, if AllowOverride is enabled throughout the entire site, the server must dig through every directory, searching for htaccess files that may not even exist. To prevent this, we disable the AllowOverride in the site’s root htaccess file and then enableAllowOverride only in required directories via the server config file (refer to this section for more information). Note: if you do not have access to your site’s server config file and also need AllowOverride privileges, do not use this directive:

# increase performance by disabling allowoverride
AllowOverride None

Improving Performance by Passing the Character Set ^

Prevent certain 500 error displays by passing the default character set parameter before you get there. Note: replace the “utf-8” below with the charset that your site is using:

# pass the default character set
AddDefaultCharset utf-8

Improving Performance by Preserving Bandwidth ^

To increase performance on PHP enabled servers, add the following directive:

# preserve bandwidth for PHP enabled servers
<ifmodule mod_php4.c>
php_value zlib.output_compression 16386
</ifmodule>

Disable the Server Signature ^

Here we are disabling the digital signature that would otherwise identify the server:

# disable the server signature
ServerSignature Off

Set the Server Timezone ^

Here we are instructing the server to synchronize chronologically according to the time zone of some specified state:

# set the server timezone
SetEnv TZ America/Washington

Set the Email Address for the Server Administrator ^

Here we are specifying the default email address for the server administrator:

# set the server administrator email
SetEnv SERVER_ADMIN default@domain.com

Improve Site Transfer Speed by Enabling File Caching ^

The htaccess genius over at askapache.com explains how to dramatically improve your site’s transfer speed by enabling file caching ³. Using time in seconds* to indicate the duration for which cached content should endure, we may generalize the htaccess rules as such (edit file types and time value to suit your needs):

# cache images and flash content for one month
<FilesMatch ".(flv|gif|jpg|jpeg|png|ico|swf)$">
Header set Cache-Control "max-age=2592000"
</FilesMatch>

# cache text, css, and javascript files for one week
<FilesMatch ".(js|css|pdf|txt)$">
Header set Cache-Control "max-age=604800"
</FilesMatch>

# cache html and htm files for one day
<FilesMatch ".(html|htm)$">
Header set Cache-Control "max-age=43200"
</FilesMatch>

# implement minimal caching during site development
<FilesMatch "\.(flv|gif|jpg|jpeg|png|ico|js|css|pdf|swf|html|htm|txt)$">
Header set Cache-Control "max-age=5"
</FilesMatch>

# explicitly disable caching for scripts and other dynamic files
<FilesMatch "\.(pl|php|cgi|spl|scgi|fcgi)$">
Header unset Cache-Control
</FilesMatch>

# alternate method for file caching
ExpiresActive On
ExpiresDefault A604800 # 1 week
ExpiresByType image/x-icon A2419200 # 1 month
ExpiresByType application/x-javascript A2419200 # 1 month
ExpiresByType text/css A2419200 # 1 month
ExpiresByType text/html A300 # 5 minutes
# disable caching for scripts and other dynamic files
<FilesMatch "\.(pl|php|cgi|spl|scgi|fcgi)$">
ExpiresActive Off
</FilesMatch>

• * Convert common time intervals into seconds:
• 300 = 5 minutes
• 2700 = 45 minutes
• 3600 = 1 hour
• 54000 = 15 hours
• 86400 = 1 day
• 518400 = 6 days
• 604800 = 1 week
• 1814400 = 3 weeks
• 2419200 = 1 month
• 26611200 = 11 months
• 29030400 = 1 year = never expires

Set the default language and character set ^

Here is an easy way to set the default language for pages served by your server (edit the language to suit your needs):

# set the default language
DefaultLanguage en-US

Likewise, here we are setting the default character set (edit to taste):

# set the default character set
AddDefaultCharset UTF-8

Declare specific/additional MIME types ^

# add various mime types
AddType application/x-shockwave-flash .swf
AddType video/x-flv .flv
AddType image/x-icon .ico

Send character set and other headers without meta tags ^

# send the language tag and default character set
# AddType 'text/html; charset=UTF-8' html
AddDefaultCharset UTF-8
DefaultLanguage en-US

Limit server request methods to GET and PUT ^

# limit server request methods to GET and PUT
Options -ExecCGI -Indexes -All
RewriteEngine on
RewriteCond %{REQUEST_METHOD} ^(TRACE|TRACK|OPTIONS|HEAD) RewriteRule .* - [F]

Selectively process files according to server request method ^

# process files according to server request method
Script PUT /cgi-bin/upload.cgi
Script GET /cgi-bin/download.cgi

Execute various file types through a cgi script ^

For those special occasions where certain file types need to be processed with some specific cgi script, let em know who sent ya:

# execute all png files via png-script.cgi
Action image/png /cgi-bin/png-script.cgi

SECURITY [ ^ ]

Prevent Access to .htaccess ^

Add the following code block to your htaccess file to add an extra layer of security. Any attempts to access the htaccess file will result in a 403 error message. Of course, your first layer of defense to protect htaccess files involves setting htaccess file permissions via CHMOD to 644:

# secure htaccess file
<Files .htaccess>
order allow,deny
deny from all
</Files>

Prevent Acess to a Specific File ^

To restrict access to a specific file, add the following code block and edit the file name, “secretfile.jpg”, with the name of the file that you wish to protect:

# prevent viewing of a specific file
<files secretfile.jpg>
order allow,deny
deny from all
</files>

Prevent acess to multiple file types ^

To restrict access to a variety of file types, add the following code block and edit the file types within parentheses to match the extensions of any files that you wish to protect:

<FilesMatch "\.(htaccess|htpasswd|ini|phps|fla|psd|log|sh)$">
Order Allow,Deny
Deny from all
</FilesMatch>

Prevent Unauthorized Directory Browsing ^

Prevent unauthorized directory browsing by instructing the server to serve a “xxx Forbidden – Authorization Required” message for any request to view a directory. For example, if your site is missing it’s default index page, everything within the root of your site will be accessible to all visitors. To prevent this, include the following htaccess rule:

# disable directory browsing
Options All -Indexes

Conversely, to enable directory browsing, use the following directive:

# enable directory browsing
Options All +Indexes

Likewise, this rule will prevent the server from listing directory contents:

# prevent folder listing
IndexIgnore *

And, finally, the IndexIgnore directive may be used to prevent the display of select file types:

# prevent display of select file types
IndexIgnore *.wmv *.mp4 *.avi *.etc

Change Default Index Page ^

This rule tells the server to search for and serve “business.html” as the default directory index. This rule must exist in the htaccess files of the root directory for which you wish to replace the default index file (e.g., “index.html”):

# serve alternate default index page
DirectoryIndex business.html

This rule is similar, only in this case, the server will scan the root directory for the listed files and serve the first match it encounters. The list is read from left to right:

# serve first available alternate default index page from series
DirectoryIndex filename.html index.cgi index.pl default.htm

Disguise Script Extensions ^

To enhance security, disguise scripting languages by replacing actual script extensions with dummy extensions of your choosing. For example, to change the “.foo” extension to “.php”, add the following line to your htaccess file and rename all affected files accordingly:

# serve foo files as php files
AddType application/x-httpd-php .foo

# serve foo files as cgi files
AddType application/x-httpd-cgi .foo

Limit Access to the Local Area Network (LAN) ^

# limit access to local area network
<Limit GET POST PUT>
order deny,allow
deny from all
allow from 192.168.0.0/33
</Limit>

Secure Directories by IP Address and/or Domain ^

In the following example, all IP addresses are allowed access except for 12.345.67.890 and domain.com:

# allow all except those indicated here
<Limit GET POST PUT>
order allow,deny
allow from all
deny from 12.345.67.890
deny from .*domain\.com.*
</Limit>

In the following example, all IP addresses are denied access except for 12.345.67.890 and domain.com:

# deny all except those indicated here
<Limit GET POST PUT>
order deny,allow
deny from all
allow from 12.345.67.890
allow from .*domain\.com.*
</Limit>

This is how to block unwanted visitors based on the referring domain. You can also save bandwidth by blocking specific file types — such as .jpg, .zip, .mp3, .mpg — from specific referring domains. Simply replace “scumbag” and “wormhole” with the offending domains of your choice:

# block visitors referred from indicated domains
<IfModule mod_rewrite.c>
RewriteEngine on
RewriteCond %{HTTP_REFERER} scumbag\.com [NC,OR]
RewriteCond %{HTTP_REFERER} wormhole\.com [NC,OR]
RewriteRule .* - [F]
</ifModule>

Prevent or allow domain access for a specified range of IP addresses ^

There are several effective ways to block a range of IP addresses via htaccess. This first method blocks an IP range specified by their CIDR (Classless Inter-Domain Routing) number. This method is useful for blocking mega-spammers such as RIPE, Optinet, and others. If, for example, you find yourself adding line after line of Apache deny directives for addresses beginning with the same first few numbers, choose one of them and try a whois lookup. Listed within the whois results will be the CIDR value representing every IP address associated with that particular network. Thus, blocking via CIDR is an effective way to eloquently prevent all IP instances of the offender from accessing your site. Here is a generalized example for blocking by CIDR (edit values to suit your needs):

# block IP range by CIDR number
<Limit GET POST PUT>
order allow,deny
allow from all
deny from 10.1.0.0/16
deny from 80.0.0/8
</Limit>

Likewise, to allow an IP range by CIDR number:

# allow IP range by CIDR number
<Limit GET POST PUT>
order deny,allow
deny from all
allow from 10.1.0.0/16
allow from 80.0.0/8
</Limit>

Another effective way to block an entire range of IP addresses involves truncating digits until the desired range is represented. As an IP address is read from left to right, its value represents an increasingly specific address. For example, a fictitious IP address of 99.88.77.66 would designate some uniquely specific IP address. Now, if we remove the last two digits (66) from the address, it would represent any address beginning with the remaining digits. That is, 99.88.77 represents 99.88.77.1, 99.88.77.2, … 99.88.77.99, …etc. Likewise, if we then remove another pair of digits from the address, its range suddenly widens to represent every IP address 99.88.x.y, where x and y represent any valid set of IP address values (i.e., you would block 256*256 = 65,536 unique IP addresses). Following this logic, it is possible to block an entire range of IP addresses to varying degrees of specificity. Here are few generalized lines exemplifying proper htaccess syntax (edit values to suit your needs):

# block IP range by address truncation
<Limit GET POST PUT>
order allow,deny
allow from all
deny from 99.88.77.66
deny from 99.88.77.*
deny from 99.88.*.*
deny from 99.*.*.*
</Limit>

Likewise, to allow an IP range by address truncation:

# allow IP range by address truncation
<Limit GET POST PUT>
order deny,allow
deny from all
allow from 99.88.77.66
allow from 99.88.77.*
allow from 99.88.*.*
allow from 99.*.*.*
</Limit>

Block or allow multiple IP addresses on one line ^

Save a little space by blocking multiple IP addresses or ranges on one line. Here are few examples (edit values to suit your needs):

# block two unique IP addresses
deny from 99.88.77.66 11.22.33.44
# block three ranges of IP addresses
deny from 99.88 99.88.77 11.22.33

Likewise, to allow multiple IP addresses or ranges on one line:

# allow two unique IP addresses
allow from 99.88.77.66 11.22.33.44
# allow three ranges of IP addresses
allow from 99.88 99.88.77 11.22.33

Miscellaneous rules for blocking and allowing IP addresses ^

Here are few miscellaneous rules for blocking various types of IP addresses. These rules may be adapted to allow the specified IP values by simply changing the deny directive to allow. Check ’em out (edit values to suit your needs):

# block a partial domain via network/netmask values
deny from 99.1.0.0/255.255.0.0

# block a single domain
deny from 99.88.77.66

# block domain.com but allow sub.domain.com
order deny,allow
deny from domain.com
allow from sub.domain.com

Stop Hotlinking, Serve Alternate Content ^

To serve ‘em some unexpected alternate content when hotlinking is detected, employ the following code, which will protect all files of the types included in the last line (add more types as needed). Remember to replace the dummy path names with real ones. Also, the name of the nasty image being served in this case is “eatme.jpe”, as indicated in the line containing theRewriteRule. Please advise that this method will also block services such as FeedBurner from accessing your images.

# stop hotlinking and serve alternate content
<IfModule mod_rewrite.c>
RewriteEngine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?domain\.com/.*$ [NC]
RewriteRule .*\.(gif|jpg)$ http://www.domain.com/eatme.jpe [R,NC,L]
</ifModule>

Note: To deliver a standard (or custom, if configured) error page instead of some nasty image of the Fonz, replace the line containing the RewriteRule in the above htaccess directive with the following line:

# serve a standard 403 forbidden error page
RewriteRule .*\.(gif|jpg)$ - [F,L]

Note: To grant linking permission to a site other than yours, insert this code block after the line containing the “domain.com” string. Remember to replace “goodsite.com” with the actual site domain:

# allow linking from the following site
RewriteCond %{HTTP_REFERER} !^http://(www\.)?goodsite\.com/.*$ [NC]

Block Evil Robots, Site Rippers, and Offline Browsers ^

Eliminate some of the unwanted scum from your userspace by injecting this handy block of code. After such, any listed agents will be denied access and receive an error message instead. Please advise that there are much more comprehensive lists available this example has been truncated for business purposes. Note: DO NOT include the “[OR]” on the very last RewriteCondor your server will crash, delivering “500 Errors” to all page requests.

# deny access to evil robots site rippers offline browsers and other nasty scum
RewriteBase /
RewriteCond %{HTTP_USER_AGENT} ^Anarchie [OR]
RewriteCond %{HTTP_USER_AGENT} ^ASPSeek [OR]
RewriteCond %{HTTP_USER_AGENT} ^attach [OR]
RewriteCond %{HTTP_USER_AGENT} ^autoemailspider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Xaldon\ WebSpider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Xenu [OR]
RewriteCond %{HTTP_USER_AGENT} ^Zeus.*Webster [OR]
RewriteCond %{HTTP_USER_AGENT} ^Zeus
RewriteRule ^.* - [F,L]

Or, instead of delivering a friendly error message (i.e., the last line), send these bad boys to the hellish website of your choice by replacing the RewriteRule in the last line with one of the following two examples:

# send em to a hellish website of your choice
RewriteRule ^.*$ http://www.hellish-website.com [R,L]

Or, to send em to a virtual blackhole of fake email addresses:

# send em to a virtual blackhole of fake email addresses
RewriteRule ^.*$ http://english-61925045732.spampoison.com [R,L]

You may also include specific referrers to your blacklist by using HTTP_REFERER. Here, we use the infamously scummy domain, “iaea.org” as our blocked example, and we use “yourdomain” as your domain (the domain to which you are blocking iaea.org):

RewriteCond %{HTTP_REFERER} ^http://www.iaea.org$
RewriteRule !^http://[^/.]\.yourdomain\.com.* - [F,L]

More Stupid Blocking Tricks ^

Note: Although these redirect techniques are aimed at blocking and redirecting nasty scumsites, the directives may also be employed for friendly redirection purposes:

# redirect any request for anything from spamsite to differentspamsite
RewriteCond %{HTTP_REFERER} ^http://.*spamsite.*$ [NC]
RewriteRule .* http://www.differentspamsite.com [R]

# redirect all requests from spamsite to an image of something at differentspamsite
RewriteCond %{HTTP_REFERER} ^http://.*spamsite.*$ [NC]
RewriteRule .* http://www.differentspamsite/something.jpg [R]

# redirect traffic from a certain address or range of addresses to another site
RewriteCond %{REMOTE_ADDR} 192.168.10.*
RewriteRule .* http://www.differentspamsite.com/index.html [R]

Even More Scum-Blocking Tricks ^

Here is a step-by-step series of code blocks that should equip you with enough knowledge to block any/all necessary entities. Read through the set of code blocks, observe the patterns, and then copy, combine and customize to suit your specific scum-blocking needs:

# set variables for user agents and referers and ip addresses
SetEnvIfNoCase User-Agent ".*(user-agent-you-want-to-block|php/perl).*" BlockedAgent
SetEnvIfNoCase Referer ".*(block-this-referrer|and-this-referrer|and-this-referrer).*" BlockedReferer
SetEnvIfNoCase REMOTE_ADDR ".*(666.666.66.0|22.22.22.222|999.999.99.999).*" BlockedAddress

# set variable for any class B network coming from a given netblock
SetEnvIfNoCase REMOTE_ADDR "66.154.*" BlockedAddress

# set variable for two class B networks 198.25.0.0 and 198.26.0.0
SetEnvIfNoCase REMOTE_ADDR "198.2(5|6)\..*" BlockedAddress

# deny any matches from above and send a 403 denied
<Limit GET POST PUT>
order deny,allow
deny from env=BlockedAgent
deny from env=BlockedReferer
deny from env=BlockedAddress
allow from all
</Limit>

Password-Protect Directories ^

Here is an excellent online tool for generating the necessary elements for a password-protected directory:

# password protect directories
htaccess Password Generator

Password-protect Files, Directories, and More.. ^

Secure site contents by requiring user authentication for specified files and/or directories. The first example shows how to password-protect any single file type that is present beneath the directory which houses the htaccess rule. The second rule employs the FilesMatch directive to protect any/all files which match any of the specified character strings. The third rule demonstrates how to protect an entire directory. The fourth set of rules provides password-protection for all IP’s except those specified. Remember to edit these rules according to your specific needs.

# password-protect single file
<Files secure.php>
AuthType Basic
AuthName "Prompt"
AuthUserFile /home/path/.htpasswd
Require valid-user
</Files>

# password-protect multiple files
<FilesMatch "^(execute|index|secure|insanity|biscuit)*$">
AuthType basic
AuthName "Development"
AuthUserFile /home/path/.htpasswd
Require valid-user
</FilesMatch>

# password-protect the directory in which this htaccess rule resides
AuthType basic
AuthName "This directory is protected"
AuthUserFile /home/path/.htpasswd
AuthGroupFile /dev/null
Require valid-user

# password-protect directory for every IP except the one specified
# place in htaccess file of a directory to protect that entire directory
AuthType Basic
AuthName "Personal"
AuthUserFile /home/path/.htpasswd
Require valid-user
Allow from 99.88.77.66
Satisfy Any

Require SSL (Secure Sockets Layer) ^

Here is an excellent method for requiring SSL (via askapache.com ³):

# require SSL
SSLOptions +StrictRequire
SSLRequireSSL
SSLRequire %{HTTP_HOST} eq "domain.tld"
ErrorDocument 403 https://domain.tld

# require SSL without mod_ssl
RewriteCond %{HTTPS} !=on [NC]
RewriteRule ^.*$ https://%{SERVER_NAME}%{REQUEST_URI} [R,L]

Automatically CHMOD Various File Types ^

This method is great for ensuring the CHMOD settings for various file types. Employ the following rules in the root htaccess file to affect all specified file types, or place in a specific directory to affect only those files (edit file types according to your needs):

# ensure CHMOD settings for specified file types
# remember to never set CHMOD 777 unless you know what you are doing
# files requiring write access should use CHMOD 766 rather than 777
# keep specific file types private by setting their CHMOD to 400
chmod .htpasswd files 640
chmod .htaccess files 644
chmod php files 600

Disguise all file extensions ^

This method will disguise all file types (i.e., any file extension) and present them as .php files (or whichever extension you choose):

# diguise all file extensions as php
ForceType application/x-httpd-php

Protect against denial-of-service (DOS) attacks by limiting file upload size ^

One method to help protect your server against DOS attacks involves limiting the maximum allowable size for file uploads. Here, we are limiting file upload size to 10240000 bytes, which is equivalent to around 10 megabytes. For this rule, file sizes are expressed in bytes. Checkhere for help with various file size conversions. Note: this code is only useful if you actually allow users to upload files to your site.

# protect against DOS attacks by limiting file upload size
LimitRequestBody 10240000

Secure directories by disabling execution of scripts ^

Prevent malicious brainiacs from actively scripting secure directories by adding the following rules to the representative htaccess file (edit file types to suit your needs):

# secure directory by disabling script execution
AddHandler cgi-script .php .pl .py .jsp .asp .htm .shtml .sh .cgi
Options -ExecCGI

USABILITY TRICKS [ ^ ]

Minimize CSS Image Flicker in IE6 ^

Add the following htaccess rules to minimize or even eliminate CSS background-image “flickering” in MSIE6:

# minimize image flicker in IE6
ExpiresActive On
ExpiresByType image/gif A2592000
ExpiresByType image/jpg A2592000
ExpiresByType image/png A2592000

Deploy Custom Error Pages ^

Replicate the following patterns to serve your own set of custom error pages. Simply replace the “/errors/###.html” with the correct path and file name. Also change the “###” preceding the path to summon pages for other errors. Note: your custom error pages must be larger than 512 bytes in size or they will be completely ignored by Internet Explorer:

# serve custom error pages
ErrorDocument 400 /errors/400.html
ErrorDocument 401 /errors/401.html
ErrorDocument 403 /errors/403.html
ErrorDocument 404 /errors/404.html
ErrorDocument 500 /errors/500.html

Provide a Universal Error Document ^

# provide a universal error document
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^.*$ /dir/error.php [L]

Employ Basic URL Spelling Check ^

This bit of voodoo will auto-correct simple spelling errors in the URL:

# automatically corect simple speling erors
<IfModule mod_speling.c>
CheckSpelling On
</IfModule>

Instruct browser to download multimedia files rather than display them ^

Here is a useful method for delivering multimedia file downloads to your users. Typically, browsers will attempt to play or stream such files when direct links are clicked. With this method, provide a link to a multimedia file and a dialogue box will provide users the choice of saving the file or opening it. Here are a few htaccess rules demonstrating the technique (edit file types according to your specific needs):

# instruct browser to download multimedia files
AddType application/octet-stream .avi
AddType application/octet-stream .mpg
AddType application/octet-stream .wmv
AddType application/octet-stream .mp3

Instruct server to display source code for dynamic file types ^

There are many situations where site owners may wish to display the contents of a dynamic file rather than executing it as a script. To exercise this useful technique, create a directory in which to place dynamic files that should be displayed rather than executed, and add the following line of code to the htaccess file belonging to that directory. This method is known to work for .pl,.py, and .cgi file-types. Here it is:

RemoveHandler cgi-script .pl .py .cgi

Redirect visitors to a temporary site during site development ^

During web development, maintenance, or repair, send your visitors to an alternate site while retaining full access for yourself. This is a very useful technique for preventing visitor confusion or dismay during those awkward, web-development moments. Here are the generalized htaccess rules to do it (edit values to suit your needs):

# redirect all visitors to alternate site but retain full access for you
ErrorDocument 403 http://www.alternate-site.com
Order deny,allow
Deny from all
Allow from 99.88.77.66

Provide a password prompt for visitors during site development ^

Here is another possible solution for “hiding” your site during those private, site-under-construction moments. Here we are instructing Apache to provide visitors with a password prompt while providing open access to any specifically indicated IP addresses or URL’s. Edit the following code according to your IP address and other development requirements (thanks to Caleb at askapache.com for sharing this trick ³):

# password prompt for visitors
AuthType basic
AuthName "This site is currently under construction"
AuthUserFile /home/path/.htpasswd
AuthGroupFile /dev/null
Require valid-user
# allow webmaster and any others open access
Order Deny,Allow
Deny from all
Allow from 111.222.33.4
Allow from favorite.validation/services/
Allow from googlebot.com
Satisfy Any

Prevent file or directory access according to specified time periods ^

Prevent viewing of all pictures of Fonzi during the midnight hour — or any files during any time period — by using this handy htaccess ruleset:

# prevent access during the midnight hour
RewriteCond %{TIME_HOUR} ^12$
RewriteRule ^.*$ - [F,L]

# prevent access throughout the afternoon
RewriteCond %{TIME_HOUR} ^(12|13|14|15)$
RewriteRule ^.*$ - [F,L]

REDIRECT TRICKS [ ^ ]

Important Note About Redirecting via mod_rewrite ^

For all redirects using the mod_rewrite directive, it is necessary to have the RewriteEngineenabled. It is common practice to enable the mod_rewrite directive in either the server configuration file or at the top of the site’s root htaccess file. If the mod_rewrite directive is not included in either of these two places, it should be included as the first line in any code block that utilizes a rewrite function (i.e., mod_rewrite), but only needs to be included once for each htaccess file. The proper mod_rewrite directive is included here for your convenience, but may or may not also be included within some of the code blocks provided in this article:

# initialize and enable rewrite engine
RewriteEngine on

Redirect from http://www.domain.com to http://domain.com ^

This method uses a “301 redirect” to establish a permanent redirect from the “www-version” of a domain to its respectively corresponding “non-www version”. Be sure to test immediately after preparing 301 redirects and remove it immediately if any errors occur. Use a “server header checker” to confirm a positive 301 response. Further, always include a trailing slash “/” when linking directories. Finally, be consistent with the “www” in all links (either use it always or never).

# permanently redirect from www domain to non-www domain
RewriteEngine on
Options +FollowSymLinks
RewriteCond %{HTTP_HOST} ^www\.domain\.tld$ [NC]
RewriteRule ^(.*)$ http://domain.tld/$1 [R=301,L]

Redirect from http://old-domain.com to http://new-domain.com ^

For a basic domain change from “old-domain.com” to “new-domain.com” (and folder/file names have not been changed), use the Rewrite rule to remap the old domain to the new domain. When checking the redirect live, the old domain may appear in the browser’s address bar. Simply check an image path (right-click an image and select “properties”) to verify proper redirection. Remember to check your site thoroughly after implementing this redirect.

# redirect from old domain to new domain
RewriteEngine On
RewriteRule ^(.*)$ http://www.new-domain.com/$1 [R=301,L]

Redirect String Variations to a Specific Address ^

For example, if we wanted to redirect any requests containing the character string, “perish”, to our main page at http://perishablepress.com/, we would replace “some-string” with “perish” in the following code block:

# redirect any variations of a specific character string to a specific address
RewriteRule ^some-string http://www.domain.com/index.php/blog/target [R]

Here are two other methods for accomplishing string-related mapping tasks:

# map URL variations to the same directory on the same server
AliasMatch ^/director(y|ies) /www/docs/target

# map URL variations to the same directory on a different server
RedirectMatch ^/[dD]irector(y|ies) http://domain.com

Other Fantastic Redirect Tricks ^

Redirect an entire site via 301:

# redirect an entire site via 301
redirect 301 / http://www.domain.com/

Redirect a specific file via 301:

# redirect a specific file via 301
redirect 301 /current/currentfile.html http://www.newdomain.com/new/newfile.html

Redirect an entire site via permanent redirect:

# redirect an entire site via permanent redirect
Redirect permanent / http://www.domain.com/

Redirect a page or directory via permanent redirect:

# redirect a page or directory
Redirect permanent old_file.html http://www.new-domain.com/new_file.html
Redirect permanent /old_directory/ http://www.new-domain.com/new_directory/

Redirect a file using RedirectMatch:

# redirect a file using RedirectMatch
RedirectMatch 301 ^.*$ http://www.domain.com/index.html

Note: When redirecting specific files, use Apache‘s Redirect rule for files within the same domain. Use Apache‘s RewriteRule for any domains, especially if they are different. TheRewriteRule is more powerful than the Redirect rule, and thus should serve you more effectively.

Thus, use the following for a stronger, harder page redirection (first line redirects a file, second line a directory, and third a domain):

# redirect files directories and domains via RewriteRule
RewriteRule http://old-domain.com/old-file.html http://new-domain.com/new-file.html
RewriteRule http://old-domain.com/old-dir/ http://new-domain.com/new-dir/
RewriteRule http://old-domain.com/ http://new-domain.com/

Send visitors to a subdomain ^

This rule will ensure that all visitors are viewing pages via the subdomain of your choice. Edit the “subdomain”, “domain”, and “tld” to match your subdomain, domain, and top-level domain respectively:

# send visitors to a subdomain
RewriteCond %{HTTP_HOST} !^$
RewriteCond %{HTTP_HOST} !^subdomain\.domain\.com$ [NC]
RewriteRule ^/(.*)$ http://subdomain.domain.tld/$1 [L,R=301]

More fun with RewriteCond and RewriteRule ^

# rewrite only if the file is not found
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.+)special\.html?$ cgi-bin/special/special-html/$1

# rewrite only if an image is not found
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule images/special/(.*).gif cgi-bin/special/mkgif?$1

# seo-friendly rewrite rules for various directories
RewriteRule ^(.*)/aud/(.*)$ $1/audio-files/$2 [L,R=301]
RewriteRule ^(.*)/img/(.*)$ $1/image-files/$2 [L,R=301]
RewriteRule ^(.*)/fla/(.*)$ $1/flash-files/$2 [L,R=301]
RewriteRule ^(.*)/vid/(.*)$ $1/video-files/$2 [L,R=301]

# broswer sniffing via htaccess environmental variables
RewriteCond %{HTTP_USER_AGENT} ^Mozilla.*
RewriteRule ^/$ /index-for-mozilla.html [L]
RewriteCond %{HTTP_USER_AGENT} ^Lynx.*
RewriteRule ^/$ /index-for-lynx.html [L]
RewriteRule ^/$ /index-for-all-others.html [L]

# redirect query to Google search
Options +FollowSymlinks
RewriteEngine On
RewriteCond %{REQUEST_URI} .google\.php*
RewriteRule ^(.*)$ ^http://www.google.com/search?q=$1 [R,NC,L]

# deny request according to the request method
RewriteCond %{REQUEST_METHOD} ^(TRACE|TRACK|OPTIONS|HEAD)$ [NC]
RewriteRule ^.*$ - [F]

# redirect uploads to a better place
RewriteCond %{REQUEST_METHOD} ^(PUT|POST)$ [NC]
RewriteRule ^(.*)$ /cgi-bin/upload-processor.cgi?p=$1 [L,QSA]

More fun with Redirect 301 and RedirectMatch 301 ^

# seo friendly redirect for a single file
Redirect 301 /old-dir/old-file.html http://domain.com/new-dir/new-file.html

# seo friendly redirect for multiple files
# redirects all files in dir directory with first letters xyz
RedirectMatch 301 /dir/xyz(.*) http://domain.com/$1

# seo friendly redirect entire site to a different domain
Redirect 301 / http://different-domain.com

WORDPRESS TRICKS [ ^ ]

Secure WordPress Contact Forms ^

Protect your insecure WordPress contact forms against online unrighteousness by verifying the domain from whence the form is called. Remember to replace the “domain.com” and “contact.php” with your domain and contact-form file names, respectively.

# secure wordpress contact forms via referrer check
RewriteCond %{HTTP_REFERER} !^http://www.domain.com/.*$ [NC]
RewriteCond %{REQUEST_POST} .*contact.php$
RewriteRule .* - [F]

WordPress Permalinks ^

In our article, The htaccess rules for all WordPress Permalinks, we revealed the precise htaccess directives used by the WordPress blogging platform for permalink functionality. Here, for the sake of completeness, we repeat the directives only. For more details please refer to the original article:

If WordPress is installed in the site’s root directory, WordPress creates and uses the following htaccess directives:

# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>
# END WordPress

If WordPress is installed in some subdirectory “foo”, WordPress creates and uses the following htaccess directives:

# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /foo/
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /foo/index.php [L]
</IfModule>
# END WordPress

RANDOM TRICKS [ ^ ]

Activate SSI for HTML/SHTML file types: ^

# activate SSI for HTML and or SHTML file types
AddType text/html .html
AddType text/html .shtml
AddHandler server-parsed .html
AddHandler server-parsed .shtml
AddHandler server-parsed .htm

Grant CGI access in a specific directory: ^

# grant CGI access in a specific directory
Options +ExecCGI
AddHandler cgi-script cgi pl
# to enable all scripts in a directory use the following
SetHandler cgi-script

Disable magic_quotes_gpc for PHP enabled servers: ^

# turn off magic_quotes_gpc for PHP enabled servers
<ifmodule mod_php4.c>
php_flag magic_quotes_gpc off
</ifmodule>

Enable MD5 digests: ^

Note: enabling this option may result in a relative decrease in server performance.

# enable MD5 digests via ContentDigest
ContentDigest On

Expression Engine Tricks: ^

# send Atom and RSS requests to the site docroot to be rewritten for ExpressionEngine
RewriteRule .*atom.xml$ http://www.yoursite.com/index.php/weblog/rss_atom/ [R]
RewriteRule .*rss.xml$ http://www.yoursite.com/index.php/weblog/rss_2.0/ [R]

# cause all requests for index.html to be rewritten for ExpressionEngine
RewriteRule /.*index.html$ http://www.domain.com/index.php [R]

REFERENCES

• ¹ Wikipedia htaccess Resource
• ² Apache Cookbook
• ³ Ultimate htaccess Article
• More on regular expressions
• Apache htaccess Reference
• Apache htaccess Tutorial
• Apache mod_rewrite
• htaccess Forum
• Behind the Scenes with htaccess
• Automatic htaccess file generator

Introduction

In many cases using MRTG in a basic configuration to monitor the volume of network traffic to your server isn’t enough. You may also want to see graphs of CPU, disk, and memory usage. This chapter explains how to find the values you want to monitor in the SNMP MIB files and then how to use this information to configure MRTG.

All the chapter’s examples assume that the SNMP Read Only string is craz33guy and that the net-snmp-utils RPM package is installed (see Chapter 22, “ Monitoring Server Performance“).

Locating And Viewing The Contents Of Linux MIBs

Residing in memory, MIBs are data structures that are constantly updated via the SNMP daemon. The MIB configuration text files are located on your hard disk and loaded into memory each time SNMP restarts.

You can easily find your Fedora Linux MIBs by using the locate command and filtering the output to include only values with the word “snmp” in them. As you can see in this case, the MIBs are located in the /usr/share/snmp/mibs directory:

[root@bigboy tmp]# locate mib | grep snmp
/usr/share/doc/net-snmp-5.0.6/README.mib2c
/usr/share/snmp/mibs
/usr/share/snmp/mibs/DISMAN-SCHEDULE-MIB.txt
...
...
[root@bigboy tmp]#

As the MIB configurations are text files you can search for keywords in them using the grep command. This examples searches for the MIBs that keep track of TCP connections and returns the RFC1213 and TCP MIBs as the result.

[root@silent mibs]# grep -i tcp /usr/share/snmp/mibs/*.txt | grep connections
...
RFC1213-MIB.txt: "The limit on the total number of TCP connections
RFC1213-MIB.txt: "The number of times TCP connections have made a
...
TCP-MIB.txt:     "The number of times TCP connections have made a
...
...
[root@silent mibs]#

You can use the vi editor to look at the MIBs. Don’t change them, because doing so could cause SNMP to fail. MIBs are very complicated, but fortunately the key sections are commented.

Each value tracked in a MIB is called an object and is often referred to by its object ID or OID. In this snippet of the RFC1213-MIB.txt file, you can see that querying the tcpActiveOpens object returns the number of active open TCP connections to the server. The SYNTAX field shows that this is a counter value.

MIBs usually track two types of values. Counter values are used for items that continuously increase as time passes, such as the amount of packets passing through a NIC or amount of time CPU been busy since boot time. Integer values change instant by instant and are useful for tracking such statistics as the amount of memory currently being used.

tcpActiveOpens OBJECT-TYPE
    SYNTAX  Counter
    ACCESS  read-only
    STATUS  mandatory
    DESCRIPTION
            "The number of times TCP connections have made a
            direct transition to the SYN-SENT state from the
            CLOSED state."
    ::= { tcp 5 }

You’ll explore the differences between SNMP and MRTG terminologies in more detail later. Understanding them will be important in understanding how to use MRTG to track MIB values.

Testing Your MIB Value

Once you have identified an interesting MIB value for your Linux system you can then use the snmpwalk command to poll it. Many times the text aliases in a MIB only reference the OID branch and not the OID the data located in a leaf ending in an additional number like a “.0″ or “.1″. The snmpget command doesn’t work with branches giving an error stating that the MIB variable couldn’t be found.

In the example below, the ssCpuRawUser OID alias was found to be interesting, but the snmpget command fails to get a value. Follow up with the snmpwalk command shows that the value is located in ssCpuRawUser.0 instead. The snmpget is then successful in retrieving the “counter32″ type data with a current value of 396271.

[root@bigboy tmp]# snmpget -v1 -c craz33guy localhost ssCpuRawUser
Error in packet
Reason: (noSuchName) There is no such variable name in this MIB.
Failed object: UCD-SNMP-MIB::ssCpuRawUser
[root@bigboy tmp]#

[root@bigboy tmp]# snmpwalk -v1 -c craz33guy localhost ssCpuRawUser
UCD-SNMP-MIB::ssCpuRawUser.0 = Counter32: 396241
[root@bigboy tmp]# snmpget -v1 -c craz33guy localhost ssCpuRawUser.0
UCD-SNMP-MIB::ssCpuRawUser.0 = Counter32: 396271
[root@bigboy tmp]#

The MIB values that work successfully with snmpget are the ones you should use with MRTG.

Differences In MIB And MRTG Terminology

Always keep in mind that MRTG refers to MIB counter values as counter values. It refers to MIB integer and gauge values as gauge. By default, MRTG considers all values to be counters.

MRTG doesn’t plot counter values as a constantly increasing graph, it plots only how much the value has changed since the last polling cycle. CPU usage is typically tracked by MIBs as a counter value; fortunately, you can edit your MRTG configuration file to make it graph this information in a percentage use format (more on this later).

The syntax type, the MIB object name, and the description of what it does are the most important things you need to know when configuring MRTG; I’ll come back to these later.

The CPU And Memory Monitoring MIB

The UCD-SNMP-MIB MIB keeps track of a number of key performance MIB objects, including the commonly used ones in Table 23-1.

Table 23-1 Important Objects In The UCD-SNMP-MIB MIB

The TCP/IP Monitoring MIB

The TCP-MIB MIB keeps track of data connection information and contains the very useful tcpActiveOpens and tcpCurrEstab objects. Table 23.2 details the most important objects in TCP-MIB.

Table 23-2 Important Objects In The TCP-MIB MIB

Manually Configuring Your MRTG File

The MRTG cfgmaker program creates configuration files for network interfaces only, simultaneously tracking two OIDs: the NIC’s input and output data statistics. The mrtg program then uses these configuration files to determine the type of data to record in its data directory. The indexmaker program also uses this information to create the overview, or Summary View Web page for the MIB OIDs you’re monitoring.

This Summary View page shows daily statistics only. You have to click on the Summary View graphs to get the Detailed View page behind it with the daily, weekly, monthly, and annual graphs. Some of the parameters in the configuration file refer to the Detailed View, others refer to the Summary View.

If you want to monitor any other pairs of OIDs, you have to manually create the configuration files, because cfgmaker isn’t aware of any OIDs other than those related to a NIC. The mrtg and indexmaker program can be fed individual OIDs from a customized configuration file and will function as expected if you edit the file correctly.

Parameter Formats

MRTG configuration parameters are always followed by a graph name surrounded by square brackets and a colon. The format looks like this:

Parameter[graph name]: value

For ease of editing, the parameters for a particular graph are usually grouped together. Each graph can track two OIDS listed in the Target parameter, which is usually placed at the very top of the graph name list. The two OID values are separated by an & symbol; the first one can be is the input OID, and the second one is the output OID.

Legend Parameters

On the Detailed View Web page, each graph has a legend that shows the max, average, and current values of the graph’s OID statistics. You can use the legendI parameter for the description of the input graph (first graph OID) and the legendO for the output graph (second graph OID).

The space available under each graph’s legend is tiny so MRTG also has legend1 and legend2 parameters that are placed at the very bottom of the page to provide more details. Parameter legend1 is the expansion of legendI, and legend2 is the expansion of legendO.

The Ylegend is the legend for the Y axis, the value you are trying to compare. In the case of a default MRTG configuration this would be the data flow through the interface in bits or bytes per second. Here is an example of the legends of a default MRTG configuration:

YLegend[graph1]: Bits per second
Legend1[graph1]: Incoming Traffic in Bits per Second
Legend2[graph1]: Outgoing Traffic in Bits per Second
LegendI[graph1]: In
LegendO[graph1]: Out

You can prevent MRTG from printing the legend at the bottom of the graph by leaving the value of the legend blank like this:

LegendI[graph1]:

Later you’ll learn how to match the legends to the OIDs for a variety of situations.

Options Parameters

Options parameters provide MRTG with graph formatting information. The growright option makes sure the data at the right of the screen is for the most current graph values. This usually makes the graphs more intuitively easy to read. MRTG defaults to growing from the left.

The nopercent option prevents MRTG from printing percentage style statistics in the legends at the bottom of the graph. The gauge option alerts MRTG to the fact that the graphed values are of the gauge type. If the value you are monitoring is in bytes, then you can convert the output to bits using the bits option. Likewise, you can convert per second values to per minute graphs using the perminute option. Here are some examples for two different graphs:

options[graph1]: growright,nopercent,perminute

options[graph2]: gauge,bits

If you place this parameter at the top with a label of [_] it gets applied to all the graphs defined in the file. Here’s an example.

options[_]: growright

Title Parameters

The title on the Summary Page is provided by the Title parameter, the PageTop parameter tells the title for the Detailed View page. The PageTop string must start with < H1 > and end with < H1 >.

Title[graph1]: Interface eth0

PageTop[graph1]: < H1 >Detailed Statistics For Interface eth0 < H1 >

Scaling Parameters

The MaxBytes parameter is the maximum amount of data MRTG will plot on a graph. Anything more than this seems to disappear over the edge of the graph.

MRTG also tries to adjust its graphs so that the largest value plotted on the graph is always close to the top. This is so even if you set the MaxBytes parameter.

When you are plotting a value that has a known maximum and you always want to have this value at the top of the vertical legend, you may want to turn off MRTG’s auto scaling. If you are plotting percentage CPU usage, and the server reaches a maximum of 60%, with scaling, MRTG will have a vertical plot of 0% to 60%, so that the vertical peak is near the top of the graph image.

When scaling is off, and MaxBytes is set to 100, then the peak will be only 60% of the way up as the graph plots from 0% to 100%. The example removes scaling from the yearly, monthly, weekly, and daily views on the Detailed View page and gives them a maximum value of 100.

Unscaled[graph1]: ymwd
MaxBytes[graph1]: 100

Defining The MIB Target Parameters

As stated before, MRTG always tries to compare two MIB OID values that are defined by the Target parameter. You have to specify the two MIB OID objects, the SNMP password and the IP address of the device you are querying in this parameter, and separate them with an & character:

Target[graph1]: mib-object-1.0&mib-object-2.0:<SNMP-password>@<IP-address>

The numeric value, in this case .0, at the end of the MIB is required. The next example uses the SNMP command to return the user mode CPU utilization of a Linux server. Notice how the .0 is tagged onto the end of the output.

[root@silent mibs]# snmpwalk -v 1 -c craz33guy localhost ssCpuRawUser
UCD-SNMP-MIB::ssCpuRawUser.0 = Counter32: 926739
[root@silent mibs]#

The MRTG legends map to the MIBs listed in the target as shown in Table 23-3.

Table 23-3 Mapping MIBs To The Graph Legends

So in the example below, legend1 and legendI describe mib-object-1.0 and legend2 and legendO describe mib-object-2.0.

Target[graph1]: mib-object-1.0&mib-object-2.0:<SNMP-password>@<IP-address>

Plotting Only One MIB Value

If you want to plot only one MIB value, you can just repeat the target MIB in the definition as in the next example, which plots only mib-object-1. The resulting MRTG graph actually superimposes the input and output graphs one on top of the other.

Target[graph1]: mib-object-1.0&mib-object-1.0:<SNMP-password>@<IP-address>

Adding MIB Values Together For a Graph

You can use the plus sign between the pairs of MIB object values to add them together. The next example adds mib-object-1.0 and mib-object-3.0 for one graph and adds mib-object-2.0 and mib-object-4.0 for the other.

Target[graph1]: mib-object-1.0&mib-object-2.0:<SNMP-password>@<IP-address> + mib-object-3.0&mib-object-4.0:<SNMP-password>@<IP-address>

You can use other mathematical operators, such as subtract (-), multiply (*), and divide (%). Left and right parentheses are also valid. There must be white spaces before and after all these operators for MRTG to work correctly. If not, you’ll get oddly shaded graphs.

Sample Target: Total CPU Usage

Linux CPU usage is occupied by system processes, user mode processes, and a few processes running in nice mode. This example adds them all together in a single plot.

Target[graph1]:ssCpuRawUser.0&ssCpuRawUser.0:<SNMP-password>@<IP-address> + ssCpuRawSystem.0&ssCpuRawSystem.0:<SNMP-password>@<IP-address> + ssCpuRawNice.0&ssCpuRawNice.0:<SNMP-password>@<IP-address>

Be sure to place this command on a single line

Sample Target: Memory Usage

Here is an example for the plotting the amount of free memory versus the total RAM installed in the server. Notice that this is a gauge type variable.

Target[graph1]: memAvailReal.0&memTotalReal.0:<SNMP-password>@<IP-address>
options[graph1]: nopercent,growright,gauge

Next, plot the percentage of available memory. Notice how the mandatory white spaces separate the mathematical operators from the next target element.

Target[graph1]: ( memAvailReal.0& memAvailReal.0:<SNMP-password>@<IP-Address> ) * 100 / ( memTotalReal.0&memTotalReal.0:<SNMP-password>@<IP-Address> )
options[graph1]: nopercent,growright,gauge

Sample Target: Newly Created Connections

HTTP traffic caused by Web browsing usually consists of many very short lived connections. The tcpPassiveOpens MIB object tracks newly created connections and is suited for this type of data transfer. The tcpActiveOpens MIB object monitors new connections originating from the server. On smaller Web sites you may want to use the perminute option to make the graphs more meaningful.

Target[graph1]: tcpPassiveOpens.0& tcpPassiveOpens.0:<SNMP-password>@<IP-address>
MaxBytes[graph1]: 1000000
Options[graph1]: perminute

Sample Target: Total TCP Established Connections

Other protocols such as FTP and SSH create longer established connections while people download large files or stay logged into the server. The tcpCurrEstab MIB object measures the total number of connections in the established state and is a gauge value.

Target[graph1]: tcpCurrEstab.0&tcpCurrEstab.0:<SNMP-password>@<IP-address>
MaxBytes[graph1]: 1000000
Options[graph1]: gauge

Sample Target: Disk Partition Usage

In this example, you’ll monitor the /var and /home disk partitions on the system.

1) First use the df -k command to get a list of the partitions in use.

[root@bigboy tmp]# df -k
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/hda8               505605    128199    351302  27% /
/dev/hda1               101089     19178     76692  21% /boot
/dev/hda5              1035660    122864    860188  13% /home
/dev/hda6               505605      8229    471272   2% /tmp
/dev/hda3              3921436    890092   2832140  24% /usr
/dev/hda2              1510060    171832   1261520  73% /var
[root@bigboy tmp]#

2) Add two entries to your snmpd.conf file.

disk  /home
disk  /var

3) Restart the SNMP daemon to reload the values.

[root@bigboy tmp]# service snmpd restart

4) Use the snmpwalk command to query the the dskPercent MIB. Object dskPercent.1 refers to the first disk entry in snmpd.conf (/home), and dskPercent.2 refers to the second (/var).

[root@bigboy tmp]# snmpwalk -v 1 -c craz33guy localhost dskPercent.1
UCD-SNMP-MIB::dskPercent.1 = INTEGER: 13
[root@bigboy tmp]# snmpwalk -v 1 -c craz33guy localhost dskPercent.2
UCD-SNMP-MIB::dskPercent.2 = INTEGER: 73
[root@bigboy tmp]#

Your MRTG target for these gauge MIB objects should look like this:

Target[graph1]: dskPercent.1& dskPercent.1:<SNMP-password>@<IP-address>
options[graph1]: growright,gauge

Defining Global Variables

You have to make sure MRTG knows where the MIBs you’re using are located. The default location MRTG uses may not be valid. Specify their locations with the global LoadMIBs parameter. You must also define where the HTML files will be located; the example specifies the default Fedora MRTG HTML directory.

LoadMIBs: /usr/share/snmp/mibs/UCD-SNMP-MIB.txt, /usr/share/snmp/mibs/TCP-MIB.txt
workdir: /var/www/mrtg/

Implementing Advanced Server Monitoring

You now can combine all you have learned to create a configuration file that monitors all these variables, and then you can integrate it into the existing MRTG configuration.

A Complete Sample Configuration

Here is a sample configuration file that is used to query server localhost for CPU, memory, disk, and TCP connection information.

#
# File: /etc/mrtg/server-info.cfg
#
# Configuration file for non bandwidth server statistics
#

#
# Define global options
#

LoadMIBs: /usr/share/snmp/mibs/UCD-SNMP-MIB.txt,/usr/share/snmp/mibs/TCP-MIB.txt
workdir: /var/www/mrtg/

#
# CPU Monitoring
# (Scaled so that the sum of all three values doesn't exceed 100)
#

Target[server.cpu]:ssCpuRawUser.0&ssCpuRawUser.0:craz33guy@localhost + ssCpuRawSystem.0&ssCpuRawSystem.0:craz33guy@localhost + ssCpuRawNice.0&ssCpuRawNice.0:craz33guy@localhost
Title[server.cpu]: Server CPU Load
PageTop[server.cpu]: < H1 >CPU Load - System, User and Nice Processes< /H1 >
MaxBytes[server.cpu]: 100
ShortLegend[server.cpu]: %
YLegend[server.cpu]: CPU Utilization
Legend1[server.cpu]: Current CPU percentage load
LegendI[server.cpu]: Used
LegendO[server.cpu]:
Options[server.cpu]: growright,nopercent
Unscaled[server.cpu]: ymwd

#
# Memory Monitoring (Total Versus Available Memory)
#

Target[server.memory]: memAvailReal.0&memTotalReal.0:craz33guy@localhost
Title[server.memory]: Free Memory
PageTop[server.memory]: < H1 >Free Memory< /H1 >
MaxBytes[server.memory]: 100000000000
ShortLegend[server.memory]: B
YLegend[server.memory]: Bytes
LegendI[server.memory]: Free
LegendO[server.memory]: Total
Legend1[server.memory]: Free memory, not including swap, in bytes
Legend2[server.memory]: Total memory
Options[server.memory]: gauge,growright,nopercent
kMG[server.memory]: k,M,G,T,P,X

#
# Memory Monitoring (Percentage usage)
#
Title[server.mempercent]: Percentage Free Memory
PageTop[server.mempercent]: < H1 >Percentage Free Memory< /H1 >
Target[server.mempercent]: ( memAvailReal.0&memAvailReal.0:craz33guy@localhost ) * 100 / ( memTotalReal.0&memTotalReal.0:craz33guy@localhost )
options[server.mempercent]: growright,gauge,transparent,nopercent
Unscaled[server.mempercent]: ymwd
MaxBytes[server.mempercent]: 100
YLegend[server.mempercent]: Memory %
ShortLegend[server.mempercent]: Percent
LegendI[server.mempercent]: Free
LegendO[server.mempercent]: Free
Legend1[server.mempercent]: Percentage Free Memory
Legend2[server.mempercent]: Percentage Free Memory

#
# New TCP Connection Monitoring (per minute)
#

Target[server.newconns]: tcpPassiveOpens.0&tcpActiveOpens.0:craz33guy@localhost
Title[server.newconns]: Newly Created TCP Connections
PageTop[server.newconns]: < H1 >New TCP Connections< /H1 >
MaxBytes[server.newconns]: 10000000000
ShortLegend[server.newconns]: c/s
YLegend[server.newconns]: Conns / Min
LegendI[server.newconns]: In
LegendO[server.newconns]: Out
Legend1[server.newconns]: New inbound connections
Legend2[server.newconns]: New outbound connections
Options[server.newconns]: growright,nopercent,perminute

#
# Established TCP Connections
#

Target[server.estabcons]: tcpCurrEstab.0&tcpCurrEstab.0:craz33guy@localhost
Title[server.estabcons]: Currently Established TCP Connections
PageTop[server.estabcons]: < H1 >Established TCP Connections< /H1 >
MaxBytes[server.estabcons]: 10000000000
ShortLegend[server.estabcons]:
YLegend[server.estabcons]: Connections
LegendI[server.estabcons]: In
LegendO[server.estabcons]:
Legend1[server.estabcons]: Established connections
Legend2[server.estabcons]:
Options[server.estabcons]: growright,nopercent,gauge

#
# Disk Usage Monitoring
#

Target[server.disk]: dskPercent.1&dskPercent.2:craz33guy@localhost
Title[server.disk]: Disk Partition Usage
PageTop[server.disk]: < H1 >Disk Partition Usage /home and /var< /H1 >
MaxBytes[server.disk]: 100
ShortLegend[server.disk]: %
YLegend[server.disk]: Utilization
LegendI[server.disk]: /home
LegendO[server.disk]: /var
Options[server.disk]: gauge,growright,nopercent
Unscaled[server.disk]: ymwd

Testing The Configuration

The next step is to test that MRTG can load the configuration file correctly.

Restart SNMP to make sure the disk monitoring commands in the snmpd.conf file are activated. Run the /usr/bin/mrtg command followed by the name of the configuration file three times. If all goes well, MRTG will complain only about the fact that certain database files don’t exist. MRTG then creates the files. By the third run, all the files are created and MRTG should operate smoothly.

[root@bigboy tmp]# service snmpd restart
[root@bigboy tmp]# env LANG=C /usr/bin/mrtg /etc/mrtg/server-stats.cfg

Creating A New MRTG Index Page To Include This File

Use the indexmaker command and include your original MRTG configuration file from Chapter 22, “Monitoring Server Performance“, (/etc/mrtg/mrtg.cfg) plus the new one you created (/etc/mrtg/server-stats.cfg).

[root@bigboy tmp]# indexmaker --output=/var/www/mrtg/index.html \
/etc/mrtg/mrtg.cfg /etc/mrtg/server-stats.cfg

Configuring cron To Use The New MRTG File

The final step is to make sure that MRTG is configured to poll your server every five minutes using this new configuration file. To do so, add this line to your /etc/cron.d/mrtg file.

0-59/5 * * * * root env LANG=C /usr/bin/mrtg /etc/mrtg/server-stats.cfg

Some versions of Linux require you to edit your /etc/crontab file instead. See Chapter 22, “ Monitoring Server Performance“, for more details. You will also have to restart cron with the service crond restart for it to read its new configuration file that tells it to additionally run MRTG every five minutes using the new MRTG configuration file.

[root@bigboy tmp]# service crond restart

Monitoring Non Linux MIB Values

All the MIBs mentioned so far are for Linux systems; other types of systems will need additional MIBs whose correct installation may be unclear in user guides or just not available. In such cases, you’ll need to know the exact value of the OID.

Scenario

Imagine that your small company has purchased a second-hand Cisco switch to connect its Web site servers to the Internet. The basic MRTG configuration shown in Chapter 22, “ Monitoring Server Performance“, provides the data bandwidth statistics, but you want to measure the CPU load the traffic is having on the device, as well. Downloading MIBs from Cisco and using them with the snmpget command was not a success. You do not know what to do next. Find The OIDs

When MIB values fail, it is best to try to find the exact OID value. Like most network equipment manufacturers, Cisco has an FTP site from which you can download both MIBs and OIDs. The SNMP files for Cisco’s devices can be found at ftp.cisco.com in the /pub/mibs directory; OIDs are in the oid directory beneath that.

After looking at all the OID files, you decide that the file CISCO-PROCESS-MIB.oid will contain the necessary values and find these entries inside it.

"cpmCPUTotalPhysicalIndex"  "1.3.6.1.4.1.9.9.109.1.1.1.1.2"
"cpmCPUTotal5sec"           "1.3.6.1.4.1.9.9.109.1.1.1.1.3"
"cpmCPUTotal1min"           "1.3.6.1.4.1.9.9.109.1.1.1.1.4"
"cpmCPUTotal5min"           "1.3.6.1.4.1.9.9.109.1.1.1.1.5"
"cpmCPUTotal5secRev"        "1.3.6.1.4.1.9.9.109.1.1.1.1.6"
"cpmCPUTotal1minRev"        "1.3.6.1.4.1.9.9.109.1.1.1.1.7"
"cpmCPUTotal5minRev"        "1.3.6.1.4.1.9.9.109.1.1.1.1.8"

Testing The OIDs

As you can see, all the OIDs are a part of the same tree starting with 1.3.6.1.4.1.9.9.109.1.1.1.1. The OIDs provided may be incomplete, so it is best to use the snmpwalk command to try to get all the values below this root first.

[root@bigboy tmp]# snmpwalk -v1 -c craz33guy cisco-switch 1.3.6.1.4.1..9.9.109.1.1.1.1
SNMPv2-SMI::enterprises.9.9.109.1.1.1.1.2.1 = INTEGER: 0
SNMPv2-SMI::enterprises.9.9.109.1.1.1.1.3.1 = Gauge32: 32
SNMPv2-SMI::enterprises.9.9.109.1.1.1.1.4.1 = Gauge32: 32
SNMPv2-SMI::enterprises.9.9.109.1.1.1.1.5.1 = Gauge32: 32
[root@bigboy tmp]#

Although listed in the OID file, 1.1.1.1.6, 1.1.1.1.7, and 1.1.1.1.8 are not supported. Notice also how SNMP has determined that the first part of the OID value (1.3.6.1.4.1) in the original OID file maps to the word “enterprise”.

Next, you can use one the snmpget command to set only one of the OID values returned by snmpwalk.

[root@bigboy tmp]# snmpget -v1 -c craz33guy cisco-switch \
enterprises.9.9.109.1.1.1.1.5.1
SNMPv2-SMI::enterprises.9.9.109.1.1.1.1.5.1 = Gauge32: 33
[root@bigboy tmp]#

Success! Now you can use this OID value, enterprises.9.9.109.1.1.1.1.5.1, for your MRTG queries.

Speeding up MRTG with RRDtool

MRTG is a very useful program but it has a limitation. All the graphs and web pages are recreated each time a device is polled. This can potentially overload your MRTG server especially if you have a large number of monitored devices and the graphs take more than five minutes to generate. RRDtool is an application written by the creator of MRTG that can store general purpose data, but generates graphs on demand. Integrating MRTG with RRDtool can have very noticeable performance benefits. The example that follows will show you how to quickly implement a general purpose solution.

Scenario

The use of RRDtool is needed to reduce the load on a monitoring server that has been experiencing very sluggish performance due to the amount of MRTG graphs it has to regenerate every polling cycle.

• Due to space constraints, the RRD database needs to be located in the /var partition.
• The server has a default Apache configuration with the CGI files needed for dynamically generated content being located in the /var/www/cgi-bin directory.
• A CGI script is required that will read the new MRTG data in RRDtool format.
• The MRTG configuration file is /etc/mrtg/mrtg.cfg.

Here’s how to proceed.

Installing RRDtool

The RRDtool and RRDtool PERL module file can be downloaded from its website athttp://people.ee.ethz.ch/~oetiker/webtools/rrdtool/, but installation can be tricky as the installation program may look for certain supporting libraries in the wrong directories.

Fortunately the prerequisite rrdtool and rrdtool-perl packages now come as part of most Linux distributions. For more details on installing packages, see Chapter 6, “Installing Linux Software“).

Storing the MRTG Data in RRDtool Format

This phase of the integration process can be done in a few minutes, but the steps can be tricky:

• The first step is to add some new options to your cfgmaker command. The first indicates that MRTG should only store rrdtool formatted data, and the second defines the /var/mrtg directory in which it should be stored. For added security, the directory should be external to your web server’s document root.

--global 'LogFormat: rrdtool' --global "workdir: /var/mrtg"  --global 'IconDir: /mrtg'

Finally, you should also specify an icon directory which specifies the location of all miscellaneous MRTG web page icons. The RRD web interface script we’ll install later uses an incorrect location. The icon directory /mrtg is actually a partial URL location. In this Fedora scenario we are using the default Apache configuration which locates the MRTG icon files in the /var/www/mrtg directory. If you are using a non default Apache MRTG configuration or are using other Linux distributions or versions you may have to copy the icons to the custom directory in which the MRTG PNG format icon files are located.

The cfgmaker program is simple to use and is covered in in Chapter 22, “Monitoring Server Performance“.

• The next step is to create the data repository directory /var/mrtg and make it be owned by the apache user and process that runs the default Linux web server application.

[root@bigboy tmp]# mkdir /var/mrtg
[root@bigboy tmp]# chown apache /var/mrtg
[root@bigboy tmp]#

Note: If you are using SELinux you’ll have to change the context of this directory to match that of the /var/www/html directory so that the apache process will be able to read the database files when your CGI script needs them. These commands compare the contexts of the both directories and apply the correct set to /var/mrtg.

Please refer to Chapter 20, “ The Apache Web Server” for more details on file contexts with Apache.

[root@bigboy tmp]# ls -alZ /var/www | grep html
drwxr-xr-x  root     root     system_u:object_r:httpd_sys_content_t html
[root@bigboy tmp]# ls -alZ /var | grep mrtg
drwxr-xr-x  apache   root     root:object_r:var_t              mrtg
[root@bigboy tmp]# chcon -R -u system_u -r object_r -t httpd_sys_content_t /var/mrtg
[root@bigboy tmp]#

• We now need to test that the RRD files are being created correctly. Run MRTG using the /etc/mrtg/mrtg.cfg file as the source configuration file then test to see if the contents of the /var/mrtg directory have changed. Success!

[root@bigboy tmp]# ls /var/mrtg/
localhost_192.168.1.100.rrd
[root@bigboy tmp]#

The files are being created properly. Now we need to find a script to read the new data format and present it in a web format. This will be discussed next.

The MRTG / RRDtool Integration Script

The MRTG website recommends the script located on the mrtg-rrd website (http://www.fi.muni.cz/~kas/mrtg-rrd/) as being a good one to use. Let’s go ahead and install it.

• Download the script using wget. The site lists several versions; make sure you get the latest one.

[root@bigboy tmp]# wget ftp://ftp.linux.cz/pub/linux/people/jan_kasprzak/mrtg-rrd/mrtg-rrd-0.7.tar.gz
--12:42:12--  ftp://ftp.linux.cz/pub/linux/people/jan_kasprzak/mrtg-rrd/mrtg-rrd-0.7.tar.gz
           => `mrtg-rrd-0.7.tar.gz'
Resolving ftp.linux.cz... 147.251.48.205
Connecting to ftp.linux.cz|147.251.48.205|:21... connected.
Logging in as anonymous ... Logged in!
...
...
...
15:24:50 (53.53 KB/s) - `mrtg-rrd-0.7.tar.gz' saved [20863]
[root@bigboy tmp]# ls
mrtg-rrd-0.7.tar.gz
[root@bigboy tmp]#

• Extract the contents of the tar file.

[root@bigboy tmp]# tar -xzvf mrtg-rrd-0.7.tar.gz
mrtg-rrd-0.7/
mrtg-rrd-0.7/COPYING
mrtg-rrd-0.7/FAQ
mrtg-rrd-0.7/TODO
mrtg-rrd-0.7/Makefile
mrtg-rrd-0.7/mrtg-rrd.cgi
mrtg-rrd-0.7/ChangeLog
[root@bigboy tmp]#

• Create the /var/www/cgi-bin/mrtg directory and copy the mrtg-rrd.cgi file to it.

[root@bigboy tmp]# mkdir -p /var/www/cgi-bin/mrtg
[root@bigboy tmp]# cp mrtg-rrd-0.7/mrtg-rrd.cgi /var/www/cgi-bin/mrtg/
[root@bigboy tmp]#

• Edit the mrtg-rrd.cgi file and make it refer to the /etc/mrtg/mrtg.cfg file for its configuration details, or you can specify all the .cfg files in your /etc/mrtg directory.

#
# File: mrtg-rrd.cgi (Single File)
#

# EDIT THIS to reflect all your MRTG config files
BEGIN { @config_files = qw(/etc/mrtg/mrtg.cfg); }

#
# File: mrtg-rrd.cgi (multipl .cfg files)
#

# EDIT THIS to reflect all your MRTG config files
BEGIN { @config_files = </etc/mrtg/*.cfg>; }

• You should now be able to access your MRTG RRD graphs by visiting this URL:

http://www.my-web-site.org/cgi-bin/mrtg/mrtg-rrd.cgi

Once installed, RRDtool operates transparently with MRTG. You’ll have to remember to add the RRD statements to any new MRTG configurations and also add the configuration file to the CGI script. Our monitoring server can now breathe a little easier.

Troubleshooting

The troubleshooting techniques for advanced MRTG are similar to those mentioned in Chapter 22, “Monitoring Server Performance“, but because you have done some customizations you’ll have to go the extra mile.

• Verify the IP address and community string of the target device you intend to poll.
• Make sure you can do an SNMP walk of the target device. If not, revise your access controls on the target device and any firewall rules that may impede SNMP traffic.
• Ensure you can do an SNMP get of the specific OID value listed in your MRTG configuration file.
• Check your MRTG parameters to make sure they are correct. Gauge values defined as counter and vice versa will cause your graphs to have continuous zero values. Graph results that are eight times what you expect may have the bits parameter set.
• There are a few errors common to initial RRDtool integration.

Web messages like this where the reference to the MRTG configuration file in the CGI script was incorrect

Error: Cannot open config file: No such file or directory

“Permission Denied” web messages are usually caused by incorrect file permissions and / or SELinux contexts

Error: RRDs::graph failed, opening '/var/mrtg/localhost_192.168.1.100.rrd': Permission denied

Errors in the /var/log/httpd/errorlog file referring to files or directories that don’t exist can be caused by an incorrect IconDir statement in the MRTG configuration file.

[Wed Jan 04 15:42:13 2006] [error] [client 192.168.1.102] File does not exist: /var/www/html/var,
referer: http://bigboy/cgi-bin/mrtg/mrtg-rrd.cgi/ 

[Wed Jan 04 15:45:46 2006] [error] [client 192.168.1.102] script not found or unable to stat:
 /var/www/cgi-bin/mrtg/mrtg-l.png, referer: http://bigboy/cgi-bin/mrtg/mrtg-rrd.cgi/

Errors caused by not installing the pre-requisite RRD RPM modules rrdtool, perl-RRD-Simple and rrdtool-perl.

ERROR: could not find RRDs.pm. Use LibAdd: in mrtg.cfg to help mrtg find RRDs.pm

These quick steps should be sufficient in most cases and will reward you with a more manageable network.

Conclusion

Using the guidelines in this chapter you should be able to graph most SNMP MIB values available on any type of device. MRTG is an excellent, flexible monitoring tool and should be considered as a part of any systems administrator’s server management plans.

SOURCE: http://www.linuxhomenetworking.com/wiki/index.php/Quick_HOWTO_:_Ch23_:_Advanced_MRTG_for_Linux

# ifconfig

eth1      Link encap:Ethernet  HWaddr 00:1C:F0:BB:A7:28
inet addr:10.10.10.11  Bcast:10.10.10.255  Mask:255.255.255.0
inet6 addr: fe80::21c:f0ff:febb:a728/64 Scope:Link
UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
RX packets:470449435 errors:1 dropped:0 overruns:0 frame:0
TX packets:464084402 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2563674692 (2.3 GiB)  TX bytes:2243518951 (2.0 GiB)
Interrupt:225 Base address:0×2800

Just run the following syntax…

Make sure no such entries inside: /etc/sysconfig/network and /etc/modprobe.conf file

# echo “NETWORKING_IPV6=no” >> /etc/sysconfig/network
# echo “alias ipv6 off” >> /etc/modprobe.conf
# echo “alias net-pf-10 off” >> /etc/modprobe.conf
# reboot (your server to make affect)

Once reboot-ed, do :

# ifconfig

eth1      Link encap:Ethernet  HWaddr 00:1C:F0:BB:A7:28
inet addr:10.10.10.11  Bcast:10.10.10.255  Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
RX packets:470471884 errors:1 dropped:0 overruns:0 frame:0
TX packets:464109169 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2574513731 (2.3 GiB)  TX bytes:2255015395 (2.1 GiB)
Interrupt:225 Base address:0×2800

0) Backup your database.
You should probably be doing this already.  Now’s a good time to make sure that your backups ran.

1) Create the script.
You’ll need the correct permissions to query the database. Here’s the command.  Be sure to change <DATABASE_NAME> as it fits.

# mysql -p -e "show tables in <DATABASE_NAME>;" | \
tail --lines=+2 | \
xargs -i echo "ALTER TABLE {} ENGINE=INNODB;" > alter_table.sql

2) Run the script.

# mysql --database=<DATABASE_NAME> -p < alter_table.sql

3) Verify it by running this command in mysql:

mysql> show table status;

SOURCE

When invoked without arguments, the date command displays the current date and time. Depending on the options specified, date will set the date and time or print it in a user defined way. I’ve seen many people writing a perl script for calculating yesterday or tomorrow. Computer loves numbers but we love relative terms like 2 days ago. Luckily GNU date command is designed to handle relative date calculation.

Why use relative date formats?

• Ease of use
• To write your own scripts
• Automate task using cron (example run a job on last day of the month or Nth day of the month or 3rd Friday and so on)

First, print today’s date:
$ date
Sun Jun 17 12:17:24 CDT 2007

Now display Yesterday’s date:
$ date --date="1 days ago"
OR try:
$ date --date="yesterday"
Sat Jun 16 12:17:20 CDT 2007

Now display Tomorrow’s date:
$ date --date="-1 days ago"
Or better try:
$ date --date="next day"
Sat Jun 16 12:17:20 CDT 2007

Getting date in the future

To get tomorrow and day after tomorrow (tomorrow+N) use day word to get date in the future.

Getting date in the past

To get yesterday and earlier day in the past use string day ago:

Moving by whole years or months

You can add year and months keywords to get more accurate date:
$ date --date='2 year ago' # past
$ date --date='3 years' # go into future
$ date --date='2 days' # future
$ date --date='1 month ago' # past
$ date --date='2 months' # future

Moving date using more precise units

• You can use fortnight for 14 day
• Week for 7 days
• hour for 60 minutes
• minute for 60 seconds
• second for one second
• You can also use this / now / today keywords to stress the meaning

To print the date of this Friday:
$ date --date='this Friday'
To print the date of the day six months and 15 day
$ date --date='6 months 15 day'
To print the date of the day two months and 5 days ago:
$ date --date='2 months 5 day ago'

You can also use relative format to setup date and time. For example to set the system clock forward by 30 minutes, enter:
# date --set='+30 minutes'

To display date in epoch time:
$ date --date='1970-01-01 00:00:01 UTC +5 hours' +%s

SOURCE

Q. How do I format date to display on screen on for my scripts as per my requirements?

A. You need to use standard date command to format date or time for output or to use in a shell script.

Syntax to specify format
date +FORMAT

Task: Display date in mm-dd-yy format

Type the command as follows:
$ date +"%m-%d-%y"
Output:

02-27-07

Turn on 4 digit year display:
$ date +"%m-%d-%Y"
Just display date as mm/dd/yy format:
$ date +"%D"

Task: Display time only

Type the command as follows:
$ date +"%T"
Output:

19:55:04

Display locale’s 12-hour clock time
$ date +"%r"
Output:

07:56:05 PM

Display time in HH:MM format:
$ date +"%H-%M"

How do I save time/date format to a variable?

Simply type command as follows at a shell prompt:
$ NOW=$(date +"%m-%d-%Y")
To display a variable use echo / printf command:
$ echo $NOW
Sample shell script:

#!/bin/bash
NOW=$(date +"%m-%d-%Y")
FILE="backup.$NOW.tar.gz"
# rest of script

Complete list of FORMAT control characters supported by date command

FORMAT controls the output.It can be the combination of any one of the following:

%%

a literal %

%a

locale’s abbreviated weekday name (e.g., Sun)

%A

locale’s full weekday name (e.g., Sunday)

%b

locale’s abbreviated month name (e.g., Jan)

%B

locale’s full month name (e.g., January)

%c

locale’s date and time (e.g., Thu Mar 3 23:05:25 2005)

%C

century; like %Y, except omit last two digits (e.g., 21)

%d

day of month (e.g, 01)

%D

date; same as %m/%d/%y

%e

day of month, space padded; same as %_d

%F

full date; same as %Y-%m-%d

%g

last two digits of year of ISO week number (see %G)

%G

year of ISO week number (see %V); normally useful only with %V

%h

same as %b

%H

hour (00..23)

%I

hour (01..12)

%j

day of year (001..366)

%k

hour ( 0..23)

%l

hour ( 1..12)

%m

month (01..12)

%M

minute (00..59)

%n

a newline

%N

nanoseconds (000000000..999999999)

%p

locale’s equivalent of either AM or PM; blank if not known

%P

like %p, but lower case

%r

locale’s 12-hour clock time (e.g., 11:11:04 PM)

%R

24-hour hour and minute; same as %H:%M

%s

seconds since 1970-01-01 00:00:00 UTC

%S

second (00..60)

%t

a tab

%T

time; same as %H:%M:%S

%u

day of week (1..7); 1 is Monday

%U

week number of year, with Sunday as first day of week (00..53)

%V

ISO week number, with Monday as first day of week (01..53)

%w

day of week (0..6); 0 is Sunday

%W

week number of year, with Monday as first day of week (00..53)

%x

locale’s date representation (e.g., 12/31/99)

%X

locale’s time representation (e.g., 23:13:48)

%y

last two digits of year (00..99)

%Y

year

%z

+hhmm numeric timezone (e.g., -0400)

%:z

+hh:mm numeric timezone (e.g., -04:00)

%::z

+hh:mm:ss numeric time zone (e.g., -04:00:00)

%:::z

numeric time zone with : to necessary precision (e.g., -04, +05:30)

%Z

alphabetic time zone abbreviation (e.g., EDT)

SOURCE

See also:

• Shell Scripting: Creating report/log file names with date in filename

This is a quick walk through on how to set up domain keys on Centos 5 using sendmail. It should also be very similar for Redhat or Fedora.

First, Squid server installed (use up2date squid) and configured by adding following directives to file:
# vi /etc/squid/squid.conf

Modify or add following squid directives:
httpd_accel_host virtual
httpd_accel_port 80
httpd_accel_with_proxy on
httpd_accel_uses_host_header on
acl lan src 192.168.1.1 192.168.2.0/24
http_access allow localhost
http_access allow lan

Where,

• httpd_accel_host virtual: Squid as an httpd accelerator
• httpd_accel_port 80: 80 is port you want to act as a proxy
• httpd_accel_with_proxy on: Squid act as both a local httpd accelerator and as a proxy.
• httpd_accel_uses_host_header on: Header is turned on which is the hostname from the URL.
• acl lan src 192.168.1.1 192.168.2.0/24: Access control list, only allow LAN computers to use squid
• http_access allow localhost: Squid access to LAN and localhost ACL only
• http_access allow lan: — same as above –

Here is the complete listing of squid.conf for your reference (grep will remove all comments and sed will remove all empty lines, thanks to David Klein for quick hint ):
# grep -v "^#" /etc/squid/squid.conf | sed -e '/^$/d'

OR, try out sed (thanks to kotnik for small sed trick)
# cat /etc/squid/squid.conf | sed '/ *#/d; /^ *$/d'

Output:
hierarchy_stoplist cgi-bin ?
acl QUERY urlpath_regex cgi-bin \?
no_cache deny QUERY
hosts_file /etc/hosts
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern . 0 20% 4320
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl purge method PURGE
acl CONNECT method CONNECT
cache_mem 1024 MB
http_access allow manager localhost
http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
acl lan src 192.168.1.1 192.168.2.0/24
http_access allow localhost
http_access allow lan
http_access deny all
http_reply_access allow all
icp_access allow all
visible_hostname myclient.hostname.com
httpd_accel_host virtual
httpd_accel_port 80
httpd_accel_with_proxy on
httpd_accel_uses_host_header on
coredump_dir /var/spool/squid

Iptables configuration

Next, I had added following rules to forward all http requests (coming to port 80) to the Squid server port 3128 :
iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 80 -j DNAT --to 192.168.1.1:3128
iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 3128

Here is complete shell script. Script first configure Linux system as router and forwards all http request to port 3128 (Download the fw.proxy shell script):
#!/bin/sh
# squid server IP
SQUID_SERVER="192.168.1.1"
# Interface connected to Internet
INTERNET="eth0"
# Interface connected to LAN
LAN_IN="eth1"
# Squid port
SQUID_PORT="3128"
# DO NOT MODIFY BELOW
# Clean old firewall
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
# Load IPTABLES modules for NAT and IP conntrack support
modprobe ip_conntrack
modprobe ip_conntrack_ftp
# For win xp ftp client
#modprobe ip_nat_ftp
echo 1 > /proc/sys/net/ipv4/ip_forward
# Setting default filter policy
iptables -P INPUT DROP
iptables -P OUTPUT ACCEPT
# Unlimited access to loop back
iptables -A INPUT -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT
# Allow UDP, DNS and Passive FTP
iptables -A INPUT -i $INTERNET -m state --state ESTABLISHED,RELATED -j ACCEPT
# set this system as a router for Rest of LAN
iptables --table nat --append POSTROUTING --out-interface $INTERNET -j MASQUERADE
iptables --append FORWARD --in-interface $LAN_IN -j ACCEPT
# unlimited access to LAN
iptables -A INPUT -i $LAN_IN -j ACCEPT
iptables -A OUTPUT -o $LAN_IN -j ACCEPT
# DNAT port 80 request comming from LAN systems to squid 3128 ($SQUID_PORT) aka transparent proxy
iptables -t nat -A PREROUTING -i $LAN_IN -p tcp --dport 80 -j DNAT --to $SQUID_SERVER:$SQUID_PORT
# if it is same system
iptables -t nat -A PREROUTING -i $INTERNET -p tcp --dport 80 -j REDIRECT --to-port $SQUID_PORT
# DROP everything and Log it
iptables -A INPUT -j LOG
iptables -A INPUT -j DROP

Save shell script. Execute script so that system will act as a router and forward the ports:
# chmod +x /etc/fw.proxy
# /etc/fw.proxy
# service iptables save
# chkconfig iptables on

Start or Restart the squid:
# /etc/init.d/squid restart
# chkconfig squid on

Desktop / Client computer configuration

Point all desktop clients to your eth1 IP address (192.168.2.1) as Router/Gateway (use DHCP to distribute this information). You do not have to setup up individual browsers to work with proxies.

How do I test my squid proxy is working correctly?

See access log file /var/log/squid/access.log:
# tail -f /var/log/squid/access.log

Above command will monitor all incoming request and log them to /var/log/squid/access_log file. Now if somebody accessing a website through browser, squid will log information.

Problems and solutions

(a) Windows XP FTP Client

All Desktop client FTP session request ended with an error:
Illegal PORT command.

I had loaded the ip_nat_ftp kernel module. Just type the following command press Enter and voila!
# modprobe ip_nat_ftp

Please note that modprobe command is already added to a shell script (above).

(b) Port 443 redirection

I had block out all connection request from our router settings except for our proxy (192.168.1.1) server. So all ports including 443 (https/ssl) request denied. You cannot redirect port 443, from debian mailing list, “Long answer: SSL is specifically designed to prevent “man in the middle” attacks, and setting up squid in such a way would be the same as such a “man in the middle” attack. You might be able to successfully achive this, but not without breaking the encryption and certification that is the point behind SSL“.

Therefore, I had quickly reopen port 443 (router firewall) for all my LAN computers and problem was solved.

(c) Squid Proxy authentication in a transparent mode

You cannot use Squid authentication with a transparently intercepting proxy.

Further reading:

• How do I use Iptables connection tracking feature?
• How do I build a Simple Linux Firewall for DSL/Dial-up connection?
• Update: Forum topic discussion: Setting up a transparent proxy with Squid peering to ISP squid server
• Squid, a user’s guide
• Squid FAQ
• Transparent Proxy with Linux and Squid mini-HOWTO

Source

If you would like to have a set of web pages that are protected, requiring a username/password to gain access, this tutorial will show you how to set it up. This is geared towards the Unix Apache httpd servers used on holly, lamar, and www.colostate.edu. If you are using another web server, you’ll need to check that server’s documentation to see how to do this.

Steps to Password-protect a Directory

First, create a subdirectory in your web area. For the sake of this tutorial, I have created the “protect” directory. Set the permissions on the directory so that the server has read/execute. I do this by using the local command chgrp-www to set the group to the www group. This is the group that the server runs under at Colorado State University for the lamar, holly and www servers. I have used the -sd flag which sets “set group id” for a directory. This will then force any files you create within the protect directory to the www group, so if you ftp files to this directory they will be automatically readable by the server but not by any other user on the system. I then cd into the protect directory.

cd ~ric/public_html
mkdir protect
chmod g+r,g+x,o-r,o-x protect
chgrp-www -sd protect
cd protect

Next you must create a .htaccess file inside the directory you want protected. You can use either the vi or pico editors on the supported systems mentioned above or ftp the file to this directory. If you are new to unix or know little about vi then I suggest you use the pico editor or ftp the .htaccess file. The command to edit with pico is “pico .htaccess”. The .htaccess file should contain the following lines. The items in bold are things you will want to change depending on the location of the AuthUserFile and content of AuthName.

AuthUserFile /z/ric/secret/.htpasswd
AuthGroupFile /dev/null
AuthName "Ric's protected files"
AuthType Basic

<Limit GET>
require valid-user
</Limit>

The AuthName is what the user will see when they’re prompted for a password – something to the effect of “Enter the username for Ric’s Protected files”. The AuthUserFile is location of the password file and should be not accessible with a url on the server for security reasons. This is a full unix path and the permissions should be set up like the “protect” directory using the chmod and chgrp-www commands above so the only one that can read this file is the owner and the server. To get the full path of a directory, cd to that directory and enter the command “pwd” to print the working directory path.

Now you’ll have to set up the password file. You’ll need to use the htpasswd program. It is included with the Apache httpd server.

First cd to the directory that contains the password file. In this example the password file is called .htpasswd and is in the directory /z/ric/secret/ as indicated by the AuthUserFile file entry in the .htaccess file. For every username you want to add to the password file, enter the following. (the -c is only required the first time; it indicates that you want to create the .htpasswd file).

$~ cd
$~ mkdir secret
$~ cd secret
$~ htpasswd -c .htpasswd pumpkin

[ you're prompted for the password for pumpkin]
[ if you have other users enter the following. Don't use the -c]

$~ htpasswd .htpasswd user2
$~ htpasswd .htpasswd user3

Again, make sure the permissions are set up like the “protect” directory using the chmod and chgrp-www commands above so the only one that can read files in the “secret” directory is the owner and the server.

Here is the protected page using the above setup to password protect this page. The username is “pumpkin” and password is “pie”.

[source]

If you’re going to be doing a lot of Geotargeting or IP Address Lookups, please take a feed instead which will preserve both our bandwidth and your bandwidth.

Simple GET

That said, there is an easy HTTP oriented API to locate IP addresses and Geocode them. If you don’t supply the “?ip=aa.bb.cc.dd” bit, then the ip address lookup of the calling machine will be located instead (here, the aa,bb,cc,dd are decimal digits). If you add &position=true to the end of the URL then latitude and longitude will be returned also. Both HTML and XML formats are supplied for your convenience.

http://api.hostip.info/country.php
US

http://api.hostip.info/get_html.php?ip=12.215.42.19
Country: UNITED STATES (US)
City: Sugar Grove, IL

http://api.hostip.info/get_html.php?ip=12.215.42.19&position=true
Country: UNITED STATES (US)
City: Sugar Grove, IL
Latitude: 41.7696
Longitude: -88.4588

http://api.hostip.info/?ip=12.215.42.19
[use the URL above for an example - XML too long to paste below]

Country Flag

Paste the following code into your HTML to get a country flag of the ip address. The database is significantly more accurate (it ought to be 100%) for countries than for cities. It would be nice if y’all would make the flag a link to the www.hostip.info home page (http://www.hostip.info/) so they can come by if they’re interested – it’ll only benefit you in the long run. After all, the results get more accurate as more visitors submit their IP addresses!
Flag of visitor’s location:

<A HREF=”http://www.hostip.info”>
<IMG SRC=”http://api.hostip.info/flag.php” BORDER=”0″ ALT=”IP Address Lookup”>
</A>

Flag of any IP address:

<A HREF=”http://www.hostip.info”>
<IMG SRC=”http://api.hostip.info/flag.php?ip=12.215.42.19″ ALT=”IP Address Lookup”>
</A>

Embedded Applet

The following is designed to be embedded within another HTML page using the OBJECT tag. This will reproduce the zoom-in applet, (or an explanatory message with a link to fix, if the IP address lookup is unknown). Which means you can embed the applet in your own site without needing to have the local database and map data (which runs to a few gigabytes…)

All you need do is include the OBJECT block below in your HTML. Note, you can also add “?ip=aaa.bbb.ccc.ddd” to the frame.html url below to map a specific IP address.

<OBJECT DATA='http://www.hostip.info/map/frame.html'

  TYPE='text/html' BORDER=0

  WIDTH=610 HEIGHT=330 HSPACE=0 VSPACE=0>

</OBJECT>

*NIX Shell Script

You can use the following shell script to call in your favorite *NIX environment.

#!/bin/bash
lynx -dump “http://api.hostip.info/get_html.php?ip=$1″

By default, Apache comes preconfigured to serve a maximum of 256 clients simultaneously. This particular configuration setting can be found in the file /etc/httpd/conf/httpd.conf

If your server has 2 GB of RAM, and you’re sharing your server with MySQL(true in my case), you’ll want to reserve about half of it for Apache (1 GB)

MaxClients: here is the process of determining MaxClients. type

ps -U apache -u apache u

See the number of apache process running in you command prompt.

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
apache 7694 0.0 0.3 42704 6680 ? S 18:30 0:00 /usr/sbin/httpd

The above indicates that a single httpd process is using 6.6 MB of RSS (Resident Set Size) memory (or non-swapped physical memory) and that it is using 42 MB of VSZ (Virtual Size) memory. This depends on the number of modules you have loaded and running in Apache.

As shared libraries are included in this number, it’s not 100 percent accurate. We can assume that half the RSS number is “real” memory. Let’s assume that each httpd process is using (6.6/2=3.3) 4 MB of memory. So if you have 1 GB ram then divide it with 4 MB of memory, which leaves room for around 256 concurrent httpd processes.

Set MaxClients 256

Or

Somebody prefers to set MaxClients using following rule

MaxClients = 150 x RAM (GB)

So for example if you have 2 GB RAM (dedicated for apache) set this value to 300. In my case IT WILL BE 150

Or

Some individuals maintain that each httpd thread uses about 5 MB of “real” memory. So they determine by the following way..

Or

MaxClients = RAM(MB)/5

So for example if you have 2 GB RAM (dedicated for apache) set this value to 409. In my case IT WILL BE 204(1 GB for apache)

Note: There is no reason for you to set it any higher unless you have a specific problem with this value. A high value can lead to a complete server hang in case of a DOS attack. A value too low can create timeout problems for your clients if the limit is reached

StartServers – Sets the number of child server processes created on startup. This setting depends greatly on the type of webserver you run. If you run low traffic websites on that server set it low to something like 5. If you have resource intensive websites on that server you should set it close to MaxClients.

MaxRequestsPerChild – Controls the number of request the a child serves before the child is killed. This should not be set too low as it will put an unnecessary load on the apache server to recreate the child. I suggest setting it to 1000.

But we are going to use 2000 for handling heavy traffic load properly.

MinSpareServers and MaxSpareServers – MaxSpareServers and MinSpareServers control how many spare (unused) child-processes Apache will keep alive while waiting for more requests to put them to use. Each child-process consumes resources, so having MaxSpareServers set too high can cause resource problems. On the other hand, if the number of unused servers drops below MinSpareServers, Apache will fork. Leave those values to: MinSpareServers 5 MaxSpareServers 10

ServerLimit: Its better to keep Server limit same as the value of MaxClients.

MaxRequestsPerChild: I’ve Kept default apache value for this one.

So few changes need to be made in httpd.conf file which is located in /etc/httpd/conf/ directory

<IfModule prefork.c>
StartServers     140
MinSpareServers    5
MaxSpareServers   10
ServerLimit      150
MaxClients       150
MaxRequestsPerChild  4000
</IfModule>

[Note]: Response time depends on MaxClients. If you increase the MaxClients number, server will response more quickly for each request but  a high value can lead to a complete server hang.

Ab is a tool for benchmarking the performance of your Apache HyperText Transfer Protocol (HTTP) server. It does this by giving you an indication of how many requests per second your Apache installation can serve.

uptime command in your root login should not yield a load average above 1, and the server should respond to commands quickly

ab -n 10000 -c 200 -k http://your_url
-c = concurrent connections
-t = time limit
-n = # of requests

Keep tuning until you hit your maximum desired load average. For servers used interactively often, having a load above 3 is way too much to use the server comfortably. For servers used mostly as real servers, a maximum load average of 10 should be acceptable. More than that, and you’ll find yourself needing to reboot the server when experiencing heavy traffic conditions, because no terminal or remote console will respond quickly to commands, and managing the server will be impossible.

How to configure few things in php.ini file for supporting huge traffic

* Enable the compression of HTML by putting in your php.ini:

output_handler = ob_gzhandler

** Switch from file based sessions to shared memory sessions. Compile PHP with the –with-mm option and

set session.save_handler=mm

Configure mysql. Change my.cnf file for better performance.

The database parameters are tuned for systems with 1 GB RAM (for ISO CD images). If you have higher RAM, please change the following in the “my.cnf” MySQL configuration file under /etc/mysql or /etc directory.

For a machine running with 512 MB of RAM, you can set these to:

key_buffer=128M table_cache=1024 sort_buffer=64M read_buffer=2M record_buffer=4M

For a machine running with 1 GB of RAM, you can set these to:

key_buffer=256M table_cache=2048 sort_buffer=128M read_buffer=2M record_buffer=8M

For a machine running with 2 GB of RAM, you can set these to:

key_buffer=512M table_cache=3072 sort_buffer=256M read_buffer=2M record_buffer=8M

For a machine running with 4 GB of RAM, you can set these to:

key_buffer=1G table_cache=4096 sort_buffer=512M read_buffer=2M record_buffer=8M

Original Post

To search in the current directory and all sub directories for a file named httpd.conf

find . -name “httpd.conf” -print

To find some string or text, type

find . -exec grep “MaxClients” ‘{}’ \; -print

This command will search in the current directory and all sub directories. All files that contain the string with the path.

If you want to just find each file then pass it on for processing use the -q grep option. This finds the first occurrance of the search string. It then signals success to find and find continues searching for more files.

find . -exec grep -q “www.athabasca” ‘{}’ \; -print

Send 1000 Request to apache using apache benchmark

ab -n 1000 -c 200 -k YOUR_URL

To view error log of httpd. type

grep -i maxclient /var/log/httpd/error_log*

To view Process status type and load average type top and uptime respectively.

To open a file and search something(Here Example is: MaxClients) from there type

vi +/MaxClients /etc/httpd/conf/httpd.conf

To view total memory used by httpd, type

ps -ylC httpd –sort:rss

Original Post

Easy part, just create like below .htaccess file on your web folder :

AuthName “My Protected Site”
AuthUserFile /home/apache/.htpasswd
AuthType basic
Require valid-user
Order Deny,Allow
Deny from all
Allow from 192.168.1. 192.168.2.
Satisfy Any

Good luck!

We have 2 ways, to get this done. You just need to choose, which way is suitable for you

Long Way:

1. Log in to your Account Manager.

2. Visit: https://certs.godaddy.com/ManageProducts.do

3. On “Manage SSL Certificates”, click on your domain name.

4. Click on “SITE SEAL” tab to manage your site seal

5. Choose “Site Seal Image Size”

6. Click on Submit button.

7. On the right side box, copy-paste the javascript provided, into your sidebar website.

8. Done.

Easy way:

1. Log in to your Account Manager.

2. Visit Manage Site Seal page: https://certs.godaddy.com/ManageSiteSeal.do

3. Choose “Site Seal Image Size”

4. Click on Submit button.

5. On the right side box, copy-paste the javascript provided, into your sidebar website.

6. Done.

This information is ONLY relevant to PHP4 and Apache 1.3. (BUT possible can be work also in PHP 5.x and Apache 2.x ) We historically used PHP for all our web work. We have decided to migrate to ruby for lots of reasons for all our new web development but we still have lots of PHP stuff hanging around.

Background

We regularly mix PHP and SSIs for the following reasons:

• Laziness – we have a lot of historic SSI stuff lying around and do not want to change it. We prefer evolution to revolution.
• Appropriateness. Not all systems are good at everything. We find that conditionally selecting ‘lumps’ of code to deliver browser specific pages (see server side browser sniffing) is a lot cleaner and easier with SSI. That does not take away from either technology.

Nesting PHP and SSI

The rules go like this (PHP4 and Apache 1.3 – we understand that Apache 2 is more flexible but have not yet made the transition):

1. You can invoke SSI files from within PHP but must use the PHP virtual() function not include(). Variables set within PHP are NOT available to SSI so our favorite ‘wheeze’ of supplying last modified dates to a standard footer do not work.
2. You can include SSI files using the include virtual SSI directive but the SSI filename must have a .shtml extension even if the XBitHack is being used.
3. You cannot include PHP files using the include virtual SSI directive.
4. Variables set within the General Apache section (we use this technique for server side bowser sniffing) are available to both .php and .shtml files no matter how they are called.

Note: We would guess that the Apache environment for each type of file (.php and .shtml) is initialised to the same state as when the page is first called, whereas a nested .php files uses the same php environment and therefore reflects any dynamic changes.

Examples

The following is our standard level 1 template implemented in SSI first and then PHP.

SSI Version

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
<html>
<head>
<meta http-equiv="Content-Language" content="en-us">
<meta http-equiv="Content-Type" content="text/html">
<meta name="GENERATOR" content="company">
<!--#include virtual="/templates/meta.html" -->
<title>Level 1 template</title>
<!-- conditionally generated style sheet -->
<!--#include virtual="/templates/styles.shtml" -->
<!-- conditionally generated javascript code -->
<!--#include virtual="/scripts/javascript.shtml" -->
</head>
<body>
<!-- banner/page headings -->
<!--#include virtual="/templates/level_1.shtml" -->
<div class="page-content">

<!-- unique page contents go here -->

</div>
<!--#config timefmt="%B %d %Y" -->
<!--#set var="real_date" value="$LAST_MODIFIED" -->
<!--#include virtual="/templates/footer.shtml" -->
</body>
</html>

PHP Version

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
<html>
<head>
<meta http-equiv="Content-Language" content="en-us">
<meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
<meta name="GENERATOR" content="company">
<meta name="keywords" content="blah, blah">
<title>Cool Tools</title>
<?php
<!-- conditionally generated style sheet -->
  virtual ("/templates/styles.shtml");
<!-- conditionally generated javascript code -->
  virtual ("/scripts/javascript.shtml");
?>
</head>
<body>
<?php
<!-- banner/page headings -->
  virtual ("/templates/level_1.shtml");
?>
<div class="page-content">

<!-- unique page contents go here -->

</div>
<?php
  $real_date = date("F d, Y.", getlastmod());
  include ("../templates/footer.php");
?>
</body>
</html>

Notes:

1. You will notice that the styles, javascript and standard page navigation header use the PHP virtual() function because they contain SSI directives but the files are otherwise unchanged.
2. Our SSI ‘last modified’ date ‘wheeze’ for the footer does not work in a mixed PHP/SSI environment (because you cannot pass variables between PHP and SSI). Instead we have to create a “footer.php” file and set the variable ‘real_date’ using the PHP date() and getlastmod() functions. This file is invoked with the include() function because it is a standard PHP file. In ‘footer.php’ we just use ‘echo $real_date’ to place our last modified date in the output stream. Yes its simpler in PHP but now we have to maintain two versions of our standard footer.

Original

Question:

I am having troubles with serverpronto bots attacking my site in droves.

How would I block this range of ip address in .htaccess using deny:

69.60.114.0 – 69.60.125.255

for example, to block one ip I would have:
Deny from 64.251.14.99

But how would I block the whole range given?

Thank you in advance
jdMorgan

Answer:

Denying 69.60.114.0 – 69.60.125.255

Any of the following:

Deny from 64.251.114
Deny from 64.251.115
Deny from 64.251.116
Deny from 64.251.117
Deny from 64.251.118
Deny from 64.251.119
Deny from 64.251.120
Deny from 64.251.121
Deny from 64.251.122
Deny from 64.251.123
Deny from 64.251.124
Deny from 64.251.125

-or-

# Deny 69.60.114.0 – 69.60.115.255 (512 addresses)
Deny from 69.60.114.0/23
# Deny 69.60.116.0 – 69.60.119.255 (1024 addresses)
Deny from 69.60.116.0/22
# Deny 69.60.120.0 – 69.60.123.255 (1024 addresses)
Deny From 69.60.120.0/22
# Deny 69.60.124.0 – 69.60.125.255 (512 addresses)
Deny from 69.60.124.0/23

-or-

# Deny 69.60.114.0 – 69.60.115.255 (512 addresses)
Deny from 69.60.114.0/255.255.254.0
# Deny 69.60.116.0 – 69.60.119.255 (1024 addresses)
Deny from 69.60.116.0/255.255.252.0
# Deny 69.60.120.0 – 69.60.123.255 (1024 addresses)
Deny From 69.60.120.0/255.255.252.0
# Deny 69.60.124.0 – 69.60.125.255 (512 addresses)
Deny from 69.60.124.0/255.255.254.0

-or-

Setenvif Remote-Addr “^69\.60\.1(1[4-9]¦2[0-5])\.” getout
Deny from getout

How to set PHP error notice hidden in httpd.conf (vhost):

<VirtualHost *:80>
  ...
  php_flag display_startup_errors off
  php_flag display_errors off
  php_flag html_errors off
  ...
</VirtualHost>

How to set individual php.ini in httpd.conf (vhost):

<VirtualHost *:80>
  ...
  PHPIniDir '/path/to/php/conf/php-foo.ini'
  ...
</VirtualHost>

How to set individual PHPError.log in httpd.conf (vhost):

<VirtualHost *:80>
  ...
  php_flag  log_errors on
  php_value error_log  /path/to/site/PHPerror.log
  ...
</VirtualHost>

Complete Information

How to Install FFmpeg in Linux ~The Easy Way~

Original Post

FFmpeg is so important if you are planning to run a video website with streaming with conversion of video files to different video formats. This tutorial is intended for Centos/Redhat versions of Linux where any novice user can install ffmpeg without compiling the source which is a more traditional way of installing the FFmpeg software on linux servers. In this tutorial i will show you the easy way to install ffmpeg and ffmpeg-php (php extension) with just yum rather than compiling ffmpeg from source files.

FFmpeg (http://ffmpeg.mplayerhq.hu)
Mplayer + Mencoder (http://www.mplayerhq.hu/design7/dload.html)
Flv2tool (http://inlet-media.de/flvtool2)
Libogg + Libvorbis (http://www.xiph.org/downloads)
LAME MP3 Encoder (http://lame.sourceforge.net)
FlowPlayer – A Free Flash Video Player – http://flowplayer.org/

Installing FFMpeg

yum install ffmpeg ffmpeg-devel

If you get package not found, then you will need to add few lines in the yum repository for dag packages installation. Create a file named dag.repo in /etc/yum.repos.d with the following contents on it

[dag]
name=Dag RPM Repository for Red Hat Enterprise Linux
baseurl=http://apt.sw.be/redhat/el$releasever/en/$basearch/dag
gpgcheck=1
enabled=1

then

yum install ffmpeg ffmpeg-devel

If everything is fine, then the installation should proceed smoothly. If not you will get something like warning GPG public key missing .

Common Errors

To fix rpmforge GPG key warning:

rpm -Uhv http://apt.sw.be/redhat/el5/en/i386/rpmforge/RPMS/rpmforge-release-0.3.6-1.el5.rf.i386.rpm

For more information refer to this faq depending on Centos version

Missing Dependency Error:

If you get missing dependency error like shown below, in the middle of ffmpeg installation

Error: Missing Dependency: libc.so.6(GLIBC_2.4) is needed by package ffmpeg
Error: Missing Dependency: libtheora.so.0(libtheora.so.1.0) is needed by package ffmpeg
Error: Missing Dependency: rtld(GNU_HASH) is needed by package ffmpeg
Error: Missing Dependency: libc.so.6(GLIBC_2.4) is needed by package imlib2
Error: Missing Dependency: rtld(GNU_HASH) is needed by package a52dec
Error: Missing Dependency: rtld(GNU_HASH) is needed by package imlib2
Error: Missing Dependency: rtld(GNU_HASH) is needed by package gsm
Error: Missing Dependency: libc.so.6(GLIBC_2.4) is needed by package x264
Error: Missing Dependency: rtld(GNU_HASH) is needed by package xvidcore
Error: Missing Dependency: libc.so.6(GLIBC_2.4) is needed by package lame
Error: Missing Dependency: libc.so.6(GLIBC_2.4) is needed by package a52dec
Error: Missing Dependency: rtld(GNU_HASH) is needed by package faad2
Error: Missing Dependency: rtld(GNU_HASH) is needed by package x264
Error: Missing Dependency: rtld(GNU_HASH) is needed by package lame
Error: Missing Dependency: libc.so.6(GLIBC_2.4) is needed by package xvidcore
Error: Missing Dependency: libc.so.6(GLIBC_2.4) is needed by package faac
Error: Missing Dependency: libc.so.6(GLIBC_2.4) is needed by package faad2
Error: Missing Dependency: libgif.so.4 is needed by package imlib2
Error: Missing Dependency: rtld(GNU_HASH) is needed by package faac
Error: Missing Dependency: libc.so.6(GLIBC_2.4) is needed by package gsm
Error: Missing Dependency: libpng12.so.0(PNG12_0) is needed by package imlib2
Error: Missing Dependency: rtld(GNU_HASH) is needed by package libmp4v2
Error: Missing Dependency: libc.so.6(GLIBC_2.4) is needed by package libmp4v2

then most commonly you have GLIB 2.3 installed instead of GLIB 2.4 version. To check the current GLIB version installed on your server. just use:

yum list glib*

and it should list the latest GLIB package version.

The reason i was getting this error was my rpmforge packages was pointed to centos 5 versions instead of centos 4.6.

To fix dependency error:

To fix this error, you might need to check your rpmforge packages compatible to the release of your existing CentOS version.
Check the file /etc/yum.repos.d/rpmforge.repo and it should look like for Centos 4.6(Final). If you have lines like http://apt.sw.be/redhat/el5/en/mirrors-rpmforge you might need to make changes to the rpmforge.repos like shown below

Note: Backup the original rpmforge.repo file before you edit its content.

[rpmforge]
name = Red Hat Enterprise $releasever – RPMforge.net – dag
#baseurl = http://apt.sw.be/redhat/el4/en/$basearch/dag
mirrorlist = http://apt.sw.be/redhat/el4/en/mirrors-rpmforge
#mirrorlist = file:///etc/yum.repos.d/mirrors-rpmforge
enabled = 1
protect = 0
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rpmforge-dag
gpgcheck = 1

To know what linux type and version you are running

cat /etc/redhat-release

Once this is done, do again:  yum install ffmpeg.

This trick resolved the problem in my linux box running Centos 4.6 and this is the only way i found to install ffmpeg using yum.

To check the FFmpeg working:

Finally, check the ffmpeg whether it is working or not.

> ffmpeg
> ffmpeg -formats
> ffmpeg –help
// This lists path of mpeg, its modules and other path information

ffmpeg -i Input.file Output.file

To check what audi/video formats are supported

ffmpeg -formats > ffmpeg-format.txt

Open the ffmpeg-formats.txt to see the ooutput

D means decode
E means encode
V means video
A means audio
T = Truncated

Install FFMPEG-PHP Extension

FFmpeg-php is a very good extension and wrapper for PHP which can pull useful information about video through API interface. Inorder to install it you will need to download the source file and then compile and install extension in your server. You can download the source tarball : http://ffmpeg-php.sourceforge.net/

wget /path/to/this/file/ffmpeg-php-0.5.2.1.tbz2

tar -xjf ffmpeg-0.5.2.1.tbz2

phpize

./configure
make
make install

Common Errors

1. If you get command not found error for phpize, then you will need to do yum install php-devel

2. If you get error like “ffmpeg headers not found” while configuring the source.

configure: error: ffmpeg headers not found. Make sure ffmpeg is compiled as shared libraries using the –enable-shared option

then it means you have not installed ffmpeg-devel packages.

To Fix: Just install ffmpeg-devel using

yum install ffmpeg-devel

3. If you get an error like shared libraries not found problem and the program halts in the middle, then you must specify the ffmpeg installed path explicitly to the ./configure.

configure: error: ffmpeg shared libraries not found. Make sure ffmpeg is compiled as shared libraries using the –enable-shared option

To Fix:

1. First find out the ffmpeg path with ffmpeg –help command. The prefix default path should be like /usr/local/cpffmpeg
2. Configure the FFmpeg-php with –with-ffmpeg option

./configure –with-ffmpeg=/usr/local/cpffmpeg

That should resolve the problem!

Editing PHP.INI

Once you have done that without any problems then you will see the php extension file /usr/local/lib/php/extensions/no-debug-non-zts-20060613/ffmpeg.so and you will need mention that extension in php.ini file

nano /usr/local/lib/php.ini

Put the below two lines at the end of the php.ini file

[ffmpeg]
extension=ffmpeg.so

Then restart the server service httpd restart

To check whether ffmpeg enabled with php, point your browser to test.php file. It should show the confirmation of installed ffmpeg php extension

// #test.php

<?php

phpinfo()

?>

If any case the ffmpeg does not show in the phpinfo() test make sure that php.ini path to ffmpeg.so is correct. Still the problem occurs, the reason could be you might be using older versions of ffmpeg-php which is buggy. Just download the latest version of ffmpeg-php source then compile it.

Installing Mplayer + Mencoder

Just issue the following yum commands to install the rest of the packages.

yum install mplayer mencoder

Installing FlvTool2

Flvtool2 is a flash video file manipulation tool. It can calculate metadata and can cut and edit cue points for flv files.

If you are on Centos 5 try yum install flvtool2 with dag repository and if you get package not found you will need to manually download and compile the flvtool2. You can download latest version of flvtool2 here: http://rubyforge.org/projects/flvtool2/

wget <url-link>

ruby setup.rb config
ruby setup.rb setup
sudo ruby setup.rb install

If you get command not found error, it probably means that you dont have ruby installed.

yum install ruby

Thats it! Once ffmpeg works fine with php extension, download a sample video, convert to .flv format in the command line and plug it to flowplayer to see it work on your web browser. Try also to download the video file offline and see whether the converted flv file works well with both audio and video.

Useful Links

• FFmpeg (http://ffmpeg.mplayerhq.hu)
• Mplayer + Mencoder (http://www.mplayerhq.hu/design7/dload.html)
• Flv2tool (http://inlet-media.de/flvtool2)
• Libogg + Libvorbis (http://www.xiph.org/downloads)
• LAME MP3 Encoder (http://lame.sourceforge.net)
• FlowPlayer – A Free Flash Video Player – http://flowplayer.org/
• Install FFmpeg from Compiling Source (Tutorial Link)
• Nice FFmpeg Installation Tutorial (click here)
• Important Audio Codecs (http://www.mplayerhq.hu/DOCS/HTML/en/audio-codecs.html)
• Common Errors & Fixes while Installing FFmpeg (click here)

/usr/bin/ld: cannot find -lltdl
collect2: ld returned 1 exit status

What you need to do, is just follow the below steps.

1. Verify that the libtool and libtool-ltdl packages are installed.
2. Symlink libltdl.so to libltdl.so.x.x.x

If libtool and libtool-ltdl already exist, you may go to Step Two.
Step One

[root@banzaibill ~]# yum install libtool-ltdl libtool

Now you have libtool installed. To check it out, do:

[root@banzaibill ~]# yum info libtool*

Step Two

PHP looks for the libltdl library only at /usr/lib/libltdl.so

The symlink to this file is not included in the libtool packages. Do below commands:

[root@banzaibill ~]# cd /usr/lib
[root@banzaibill lib]# ln -s libltdl.so.3.1.4 libltdl.so

And that’s it. PHP should configure and compile without error.

I found good examples for this.

- Quick HOWTO (from LinuxHomeNetworking.com) – download

- Sample IPTABLES Configuration (RedHat/CentOS) – download

I declare from the beginning that I am no authority on digital certificates.

This document is a summary of all the articles I have read about openssl. It describes in short how to become your own Certificate Authority (CA) and how to create and sign your own certificate requests. Make no mistake, these certificates are good only for personal use or for use in your intranet in order to provide a secure way to login or communicate with your services, so that passwords or other data is not transmitted in the clear. Noone else will or should trust these certificates.

Prerequisites

The package openssl should be installed in the machine you will use to manage your certificates or create the certificate requests.

First things first…

The openssl package comes with some scripts that can help you create your server certificates fast, but here I will describe how to set things up from scratch in a new directory, so that you can customize things later if you like or delete everything without touching openssl’s or the system’s default files. This article is based on a Fedora installation, but will do for all distributions.

Creating the necessary directories

First of all we will create a directory tree where all certificate stuff will be kept. Fedora’s default directory is /etc/pki/tls/. So, as root, we create our own directories:

# mkdir -m 0755 /etc/pki_jungle

And then we create our CA’s directory tree:

# mkdir -m 0755 \
     /etc/pki_jungle/myCA \
     /etc/pki_jungle/myCA/private \
     /etc/pki_jungle/myCA/certs \
     /etc/pki_jungle/myCA/newcerts \
     /etc/pki_jungle/myCA/crl

• myCA is our Certificate Authority’s directory.
• myCA/certs directory is where our server certificates will be placed.
• myCA/newcerts directory is where openssl puts the created certificates in PEM (unencrypted) format and in the form cert_serial_number.pem (eg 07.pem). Openssl needs this directory, so we create it.
• myCA/crl is where our certificate revokation list is placed.
• myCA/private is the directory where our private keys are placed. Be sure that you set restrictive permissions to all your private keys so that they can be read only by root, or the user with whose priviledges a server runs. If anyone steals your private keys, then things get really bad.

Initial openssl configuration

We are going to copy the default openssl configuration file (openssl.cnf) to our CA’s directory. In Fedora, this file exists in /etc/pki/tls. So, we copy it to our CA’s dir and name it openssl.my.cnf. As root:

# cp /etc/pki/tls/openssl.cnf /etc/pki_jungle/myCA/openssl.my.cnf

This file does not need to be world readable, so we change its attributes:

# chmod 0600 /etc/pki_jungle/myCA/openssl.my.cnf

We also need to create two other files. This file serves as a database for openssl:

# touch /etc/pki_jungle/myCA/index.txt

The following file contains the next certificate’s serial number. Since we have not created any certificates yet, we set it to “01“:

# echo '01' > /etc/pki_jungle/myCA/serial

Things to remember

Here is a small legend with file extensions we will use for the created files and their meaning. All files that will be created will have one of these extensions:

• KEY – Private key (Restrictive permissions should be set on this)
• CSR – Certificate Request (This will be signed by our CA in order to create the server certificates. Afterwards it is not needed and can be deleted)
• CRT – Certificate (This can be publicly distributed)
• PEM – We will use this extension for files that contain both the Key and the server Certificate (Some servers need this). Permissions should be restrictive on these files.
• CRL – Certificate Revokation List (This can be publicly distributed)

Create the CA Certificate and Key

Now, that all initial configuration is done, we may create a self-signed certificate, that will be used as our CA’s certificate. In other words, we will use this to sign other certificate requests.

Change to our CA’s directory. This is where we should issue all the openssl commands because here is our openssl’s configuration file (openssl.my.cnf). As root:

# cd /etc/pki_jungle/myCA/

And then create your CA’s Certificate and Private Key. As root:

# openssl req -config openssl.my.cnf -new -x509 -extensions v3_ca -keyout private/myca.key -out certs/myca.crt -days 1825

This creates a self-signed certificate with the default CA extensions which is valid for 5 years. You will be prompted for a passphrase for your CA’s private key. Be sure that you set a strong passphrase. Then you will need to provide some info about your CA. Fill in whatever you like. Here is an example:

Country Name (2 letter code) [GB]:GR
State or Province Name (full name) [Berkshire]:Greece
Locality Name (eg, city) [Newbury]:Thessaloniki
Organization Name (eg, company) [My Company Ltd]:My Network
Organizational Unit Name (eg, section) []:My Certificate Authority
Common Name (eg, your name or your server's hostname) []:server.example.com
Email Address []:whatever@server.example.com

Two files are created:

• certs/myca.crt – This is your CA’s certificate and can be publicly available and of course world readable.
• private/myca.key – This is your CA’s private key. Although it is protected with a passphrase you should restrict access to it, so that only root can read it:

  # chmod 0400 /etc/pki_jungle/myCA/private/myca.key

More openssl configuration (mandatory)

Because we use a custom directory for our certificates’ management, some modifications to /etc/pki_jungle/myCA/openssl.my.cnf are necessary. Open it in your favourite text editor as root and find the following part (around line 35):

[ CA_default ]

dir     = ../../CA      # Where everything is kept
certs       = $dir/certs        # Where the issued certs are kept
crl_dir     = $dir/crl      # Where the issued crl are kept
database    = $dir/index.txt    # database index file.
#unique_subject = no            # Set to 'no' to allow creation of
                    # several ctificates with same subject.
new_certs_dir   = $dir/newcerts     # default place for new certs.

certificate = $dir/cacert.pem   # The CA certificate
serial      = $dir/serial       # The current serial number
#crlnumber  = $dir/crlnumber    # the current crl number must be
                    # commented out to leave a V1 CRL
crl     = $dir/crl.pem      # The current CRL
private_key = $dir/private/cakey.pem    # The private key
RANDFILE    = $dir/private/.rand    # private random number file

x509_extensions = usr_cert      # The extentions to add to the cert

You should modify the following settings in order to coform to our custom directory and our custom CA key and certificate:

[ CA_default ]

dir     = .                # <--CHANGE THIS
certs       = $dir/certs
crl_dir     = $dir/crl
database    = $dir/index.txt
#unique_subject = no

new_certs_dir   = $dir/newcerts

certificate = $dir/certs/myca.crt   # <--CHANGE THIS
serial      = $dir/serial
#crlnumber  = $dir/crlnumber

crl     = $dir/crl.pem
private_key = $dir/private/myca.key    # <--CHANGE THIS
RANDFILE    = $dir/private/.rand

x509_extensions = usr_cert

Create a Server certificate

Further openssl.my.cnf file’s customization is possible, so that we define our policy for certificate creation and signing or define our desired extensions for the new certificates. I may add this info to a future version of this document. It’s easy though, just try to familiarize yourself with the openssl.cnf’s structure and you’ll figure it out.

Anyway, the certificates we are going to create, without customizing openssl.my.cnf any further, are general purpose certificates and their usage in not restricted to server authentication only. One thing that you should take a note of is that the private keys will not be protected by a passphrase, so that when the services are restarted they do not ask for a passphrase. This means that you should set restrictive permissions on the private keys, so that only root or the user under whose priviledges a server runs can read these files.

Generate a Certificate Request

First, we change to our CA’s directory:

# cd /etc/pki_jungle/myCA/

Then we create the certificate request:

# openssl req -config openssl.my.cnf -new -nodes -keyout private/server.key -out server.csr -days 365

The -nodes option is needed so that the private key is not protected with a passphrase. If you do not intend to use the certificate for server authentication, you should not include it in the above command.
You can customize the number of days you want this certificate to be valid for.

You will be prompted for the certificate’s info. Here is an example:

Country Name (2 letter code) [GB]:GR
State or Province Name (full name) [Berkshire]:Greece
Locality Name (eg, city) [Newbury]:Thessaloniki
Organization Name (eg, company) [My Company Ltd]:My Network
Organizational Unit Name (eg, section) []:My Web Server
Common Name (eg, your name or your server's hostname) []:www.server.example.com
Email Address []:whatever@server.example.com

The Common Name (CN) is the info that uniquely distinguishes your service, so be sure that you type it correctly.

When prompted for some extra attributes (challenge password, optional company name) just hit the [Enter] key.
Two files are created:

• server.csr – this is the certificate request.
• private/server.key – this is the private key, which is not protected with a passphrase.

Set restrictive permissions on the private key. Only root or the user that is used to run the server should be able to read it. For example:

# chown root.root /etc/pki_jungle/myCA/private/server.key
# chmod 0400 /etc/pki_jungle/myCA/private/server.key

Or:

# chown root.apache /etc/pki_jungle/myCA/private/server.key
# chmod 0440 /etc/pki_jungle/myCA/private/server.key

Sign the Certificate Request

Now we are going to sign the certificate request and generate the server’s certificate.

First, we change to our CA’s directory:

# cd /etc/pki_jungle/myCA/

Then we sign the certificate request:

# openssl ca -config openssl.my.cnf -policy policy_anything -out certs/server.crt -infiles server.csr

You will need to supply the CA’s private key in order to sign the request. You can check the openssl.my.cnf file about what policy_anything means. In short, the fields about the Country, State or City is not required to match those of your CA’s certificate.

After all this is done two new files are created:

• certs/server.crt – this is the server’s certificate, which can be made available publicly.
• newcerts/01.pem – This is exactly the same certificate, but with the certificate’s serial number as a filename. It is not needed.

You can now delete the certificate request (server.csr). It’s no longer needed:

# rm -f /etc/pki_jungle/myCA/server.csr

Verify the certificate

You can see the certificate’s info with the following:

# openssl x509 -subject -issuer -enddate -noout -in /etc/pki_jungle/myCA/certs/server.crt

Or the following:

# openssl x509 -in certs/server.crt -noout -text

And verify that the certificate is valid for server authentication with the following:

# openssl verify -purpose sslserver -CAfile /etc/pki_jungle/myCA/certs/myca.crt /etc/pki_jungle/myCA/certs/server.crt

Server certificate and key in one file

Some servers, for example vsftpd, require that both the private key and the certificate exist in the same file. In a situation like that just do the following:

# cat certs/server.crt private/server.key > private/server-key-cert.pem

You should restrict access to the final file and delete server.crt and server.key since thay are no longer needed.

# chown root.root private/server-key-cert.pem
# chmod 0400 private/server-key-cert.pem
# rm -f certs/server.crt
# rm -f private/server.key

Revoke a Server Certificate

If you do not want a certificate to be valid any more, you have to revoke it. This is done with the command:

# openssl ca -config openssl.my.cnf -revoke certs/server.crt

Then you should generate a new CRL (Certificate Revokation List):

# openssl ca -config openssl.my.cnf -gencrl -out crl/myca.crl

The CRL file is crl/myca.crl.

Distribute your certificates and CRL

Your CA’s certificate and your servers’ certificates should be distributed to those who trust you so they can import them in their client software (web browsers, ftp clients, email clients etc). The CRL should also be published.

There are often problems with sendmail once it has been installed due to the tightening up of sendmail to stop spammers

Sendmail-8.11.6-15 Connection refused

Sendmail & tcp wrapper rejection

Cannot relay from valid ip address (Outlook)

1) Sendmail-8.11.6-15 Connection refused

Cannot telnet to port 25, then Sendmail has not been corretly set up. This is a problem with RedHat 7.3 or more where Sendmail by default is set to only send from the localhost, you could say this is Good as Sendmail can not spew when set up on a system that is not going to use it.

File: /etc/sendmail.cf

Did you make the DAEMON_OPTIONS change mentioned in the release notes? Your sendmail.cf should *NOT* have this line:

O DaemonPortOptions=Port=smtp,Addr=127.0.0.1, Name=MTA

It needs to be hashed out to this:

#O DaemonPortOptions=Port=smtp,Addr=127.0.0.1, Name=MTA

File: /etc/mail/sendmail.mc

You can also change sendmail.mc, but this is just the configuration file that is used to create sendmail.cf. You can either delete it or change the .mc file from

DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA’)

to:

dnl DAEMON_OPTIONS(`Port=smtp,Addr=127.0.0.1, Name=MTA’)

You do not need to rebuild sendmail.cf if you make the changes directly to sendmail.cf.

To rebuild sendmail.cf is a headache so better to edit sendmail.cf and restart sendmail.

2) Sendmail & tcpwrappers rejection

File: /etc/hosts.allow

sendmail: ALL EXCEPT \
203.204. \
218.

3) Cannot relay from valid ip address (Outlook)

Sendmail has been installed and the above patches have been appilied, email is being sent fine from with in Horde, but as soon as a valid client (ip address) seeks to send emails through the server using a client mail program (outlook), we get a Relaying Rejected message.

The answer to this problem was archived when researching this page on sendmail.org.
http://www.sendmail.org/~ca/email/relayingdenied.html

Feb 24 08:39:20 mail sendmail[17602]: i1NJdKCq017602: ruleset=check_rcpt, arg1=<someone@someone.co.nz>, relay=me.somehereelse.co.nz [192.168.xx.19], reject=550 5.7.1 <someone@someone.co.nz>… Relaying denied
(parts of message changed for security)

Generally the /etc/mail/access file only has allowed client ip addresses for relaying. Now with new versions of Sendmail I have found it necessary to put in the allowed name that the PCs are giving to sendmail.

File: /etc/mail/access

access-info.co.nz                 RELAY

This lines is needed in /etc/mail/access to enable name resolution.

You may also need the following line in hosts to also enable dns ip lookup

File: /etc/hosts

192.168.xx.xx    laptop.access-info.co.nz    laptop

Replace xs with valid ip address for the PC trying to send via outlook.

Original Post

p{
   color: black;
   _color: blue;
}

div#content{
    height: auto;
    min-height: 400px;
    _height: 400px;
}

In Windows Server 2003, you can use Scheduled Tasks in Control Panel to create, delete, configure, or display scheduled tasks. You can also use Schtasks.exe to schedule tasks manually.

Back to the top

Overview of the Schtasks.exe Tool

loadTOCNode(2, ’summary’);
Schtasks schedules commands and programs to run periodically or at a specific time. Schtasks adds and removes tasks from the schedule, starts and stops tasks on demand, and displays and changes scheduled tasks.

Back to the top

Syntax and Parameters

loadTOCNode(2, ’summary’);
The following is a list of the syntax and parameters that you can use with Schtasks.exe:

• Schtasks /Create

  loadTOCNode(3, ’summary’);
  Creates a new scheduled task.

  • Syntax:
    schtasks /create/tn TaskName /tr TaskRun /sc schedule [/mo modifier] [/d day] [/m month[,month...] [/i IdleTime] [/st StartTime] [/sd StartDate] [/ed EndDate] [/du duration] [/s computer [/u [domain\]user /p password]] [/ru {[Domain\]User | “System”} [/rp Password]] /?
  • Parameters:
    • /tn TaskName Specifies a name for the task.
    • /tr TaskRun Specifies the program or command that the task runs. Type the fully qualified path and file name of an executable file, script file, or batch file. If you omit the path, Schtasks.exe assumes that the file is in the Systemroot\System32 folder.
    • /sc schedule Specifies the schedule type. Valid values are MINUTE, HOURLY, DAILY, WEEKLY, MONTHLY, ONCE, ONSTART, ONLOGON, ONIDLE.
    • /mo modifier Specifies how frequently the task runs in its schedule type. This parameter is required for a MONTHLY schedule. This parameter is valid, but optional, for a MINUTE, HOURLY, DAILY, or WEEKLY schedule. The default value is 1.
    • /d day Specifies a day of the week or a day of a month. Valid only with a WEEKLY or MONTHLY schedule.
    • /m month[,month...] Specifies a month of the year. Valid values are JAN – DEC and * (every month). The /m parameter is valid only with a MONTHLY schedule. It is required when the LASTDAY modifier is used. Otherwise, it is optional and the default value is * (every month).
    • /i IdleTime Specifies how many minutes the computer is idle before the task starts. Type a whole number from 1 to 999. This parameter is valid only with an ONIDLE schedule, and then it is required.
    • /st StartTime Specifies the time of day that the task starts in HH:MM:SS 24-hour format. The default value is the current local time when the command completes. The /st parameter is valid with MINUTE, HOURLY, DAILY, WEEKLY, MONTHLY, and ONCE schedules. It is required with a ONCE schedule.
    • /sd StartDate Specifies the date that the task starts in MM/DD/YYYY format. The default value is the current date. The /sd parameter is valid with all schedules, and is required for a ONCE schedule.
    • /ed EndDate Specifies the last date that the task is scheduled to run. This parameter is optional. It is not valid in a ONCE, ONSTART, ONLOGON, or ONIDLE schedule. By default, schedules have no ending date.
    • /du Duration Specifies a maximum length of time for a minute or hourly schedule in the HHHH:MM 24-hour format. After the specified time elapses, Schtasks does not start the task again until the start time happens again. By default, task schedules have no maximum duration. This parameter is optional and valid only with a MINUTE or HOURLY schedule.
    • /s Computer Specifies the name or IP address of a remote computer, with or without backslash characters. The default is the local computer.
    • /u [domain\]user Runs the command with the permissions of the specified user account. By default, the command runs with the permissions of the user who is logged on to the computer that is running Schtasks.
    • /p password Specifies the password of the user account that you specified in the /u parameter. This parameter is required when the /u parameter is used.
    • /ru {[Domain\]User | “System”} Runs the tasks with the permission of the specified user account. By default, the task runs with the permissions of the user who is logged on to the computer that is running Schtasks.
    • /rp Password Specifies the password of the user account that is specified in the /ru parameter. If you omit this parameter when you specify a user account, Schtasks.exe prompts you for the password and obscures the text you type. Tasks that run with permissions of the NT Authority\System account do not require a password and Schtasks.exe does not prompt for one.
    • /? Displays help at the command prompt.

  back to the top

• Schtasks /Change

  loadTOCNode(3, ’summary’);
  Changes one or more of the following properties of a task:

  • The program that the task runs (/tr ).
  • The user account under which the task runs (/ru ).
  • The password for the user account (/rp ).
  • Syntax:schtasks /change /tn TaskName [/s computer [/u [domain\]user /p password]] [/tr TaskRun] [/ru [Domain\]User | “System”] [/rp Password]
  • Parameters:
    • /tn TaskName Identifies the task to be changed. Type the task name.
    • /s Computer Specifies the name or IP address of a remote computer with or without backslash characters. The default is the local computer.
    • /u [domain\]user Runs the command with the permissions of the specified user account. By default, the command runs with the permissions of the user who is logged on to the computer that is running Schtasks.
    • /p password Specifies the password of the user account that you specified in the /u parameter. This parameter is required when the /u parameter is used.
    • /tr TaskRun Changes the program that the task runs. Type the fully qualified path and file name of an executable file, script file, or batch file. If you omit the path, Schtasks.exe assumes that the file is in the Systemroot\System32 folder. The specified program replaces the original program that is run by the task.
    • /ru [Domain\]User | “System” Changes the user account for the task.
    • /rp Password Changes the account password for the task. Type the new password.
    • /? Displays help at the command prompt.

  back to the top

• Schtasks /Run

  loadTOCNode(3, ’summary’);
  Starts a scheduled task immediately. The run operation ignores the schedule, but uses the program file location, user account, and password that are saved in the task to run the task immediately.

  • Syntax:schtasks /run /tn TaskName [/s computer [/u [domain\]user /p password]] /?
  • Parameters:
    • /tn TaskName Identifies the task. This parameter is required.
    • /s Computer Specifies the name or IP address of a remote computer with or without backslash characters. The default is the local computer.
    • /u [domain\]user Runs the command with the permissions of the specified user account. By default, the command runs with the permissions of the user who it logged on to the computer that is running Schtasks.
    • /p password Specifies the password of the user account that you specified in the /u parameter. This parameter is required when the /u parameter is used.
    • /? Displays help at the command prompt.

  back to the top

• Schtasks /End

  loadTOCNode(3, ’summary’);
  Stops a program that was started by a task.

  • Syntax: schtasks /end /tn TaskName [/s computer [/u [domain\]user /p password]] /?
  • Parameters:
    • /tn TaskName Identifies the task that started the program. This parameter is required.
    • /s Computer Specifies the name or IP address of a remote computer with or without backslash characters. The default is the local computer.
    • /u [domain\]user Runs the command with the permissions of the specified user account. By default, the command runs with the permissions of the user who is logged on to the computer that is running Schtasks.
    • /p password Specifies the password of the user account that is specified in the /u parameter. This parameter is required when the /u parameter is used. /? Displays help.

  back to the top

• Schtasks /Delete

  loadTOCNode(3, ’summary’);
  Deletes a scheduled task.

  • Syntax:schtasks /delete /tn {TaskName | *} [/f ] [/s computer [/u [domain\]user/p password]] [/? ]
  • Parameters:
    • /tn {TaskName | *} Identifies the task being deleted. This parameter is required.
      • TaskName Deletes the named task.
      • * Deletes all the scheduled tasks on the computer.
    • /f Suppresses the confirmation message. The task is deleted without warning.
    • /s Computer Specifies the name or IP address of a remote computer with or without backslash characters. The default is the local computer.
    • /u [domain\]user Runs the command with the permissions of the specified user account. By default, the command runs with the permissions of the user who is logged on to the computer that is running Schtasks.
    • /p password Specifies the password of the user account that you specified in the /u parameter. This parameter is required when the /u parameter is used.
    • /? Displays help at the command prompt.

  back to the top

• Schtasks /Query

  loadTOCNode(3, ’summary’);
  Displays all the tasks that are scheduled to run on the computer, including those that are scheduled by other users:

  • Syntax:schtasks [/query] [/fo {TABLE | LIST | CSV}] [/nh ] [/v] [/s computer [/u [domain\]user/p password]]
  • Parameters:[/query] The operation name is optional. Typing schtasks without any parameters performs a query.
  • /fo {TABLE | LIST | CSV} Specifies the output format. TABLE is the default. /nh Omits column headings from the table display. This parameter is valid with the TABLE and CSV output formats.
  • /v Adds advanced properties of the tasks to the display. Queries using /v should be formatted as LIST or CSV.
  • /s Computer Specifies the name or IP address of a remote computer with or without backslash characters. The default is the local computer.
  • /u [domain\]user Runs the command with the permissions of the specified user account. By default, the command runs with the permissions of the user who is logged on to the computer that is running Schtasks.
  • /p password Specifies the password of the user account that is specified in the /u parameter. This parameter is required when the /u parameter is used.
  • /? Displays help at the command prompt.

Back to the top

How to Create a Scheduled Task

loadTOCNode(2, ’summary’);
To create a scheduled task:

1. Click Start, click Run, type cmd, and then click OK.
2. At the command prompt, type net start, and then press ENTER to display a list of currently running services. If Task Scheduler is not displayed in the list, type net start “task scheduler”, and then press ENTER.
3. At the command prompt, type schtasks /create /tn “Application_Name” /tr c:\apps\Application_Name /sc Value /st HH:MM:SS /ed MM/DD/YYYY, and then press ENTER. Note that you may have to change the parameters for your situation. For example, you might type schtasks /create /tn “My App” /tr c:\apps\myapp.exe /sc daily /st 08:00:00 /ed 12/31/2004 This example schedules the MyApp program to run once a day, every day, at 8:00 A.M. until December 31, 2004. Because it omits the /mo parameter, the default interval of 1 is used to run the command every day.

Back to the top

How to Change a Scheduled Task

loadTOCNode(2, ’summary’);
To change a scheduled task:

1. Click Start, click Run, type cmd, and then click OK.
2. At the command prompt, typenet start, and then press ENTER to display a list of currently running services. If Task Scheduler is not displayed in the list, type net start “task scheduler”, and then press ENTER.
3. At the command prompt, typeschtasks /change /tn TaskName [/s computer [/u [domain\]user /p password]] [/tr TaskRun] [/ru [Domain\]User | “System”] [/rp Password] , and then press ENTER. Note that you may have to change the parameters for your situation. For example, to change the program that a task runs, type: schtasks /change /tn “Application_Name” /tr C:\File_Path\Application_Name.exe

Back to the top

How to Run a Scheduled Task

loadTOCNode(2, ’summary’);
To manually run a scheduled task outside its schedule:

1. Click Start, click Run, type cmd, and then click OK.
2. At the command prompt, type net start, and then press ENTER to display a list of currently running services. If Task Scheduler is not displayed in the list, type net start “task scheduler”, and then press ENTER.
3. At the command prompt, type schtasks /run /tn TaskName [/s computer [/u [domain\]user /p password]] , and then press ENTER. Note that you may have to change the parameters for your situation. For example, to run a task on the local computer, type schtasks /run /tn “Task_Name” .

Back to the top

How to End a Scheduled Task

loadTOCNode(2, ’summary’);
To end a scheduled task:

1. Click Start, click Run, type cmd, and then click OK.
2. At the command prompt, type net start, and then press ENTER to display a list of currently running services. If Task Scheduler is not displayed in the list, type net start “task scheduler”, and then press ENTER.
3. At the command prompt, type schtasks /end /tn TaskName [/s computer [/u [domain\]user /p password]] , and then press ENTER. For example, to end the instances of a program that was started by a scheduled task on a local computer, type schtasks /end /tn “Task_Name“.

Back to the top

How to Delete a Scheduled Task

loadTOCNode(2, ’summary’);
To delete a scheduled task:

1. Click Start, click Run, type cmd, and then click OK.
2. At the command prompt, type net start, and then press ENTER to display a list of currently running services. If Task Scheduler is not displayed in the list, type net start “task scheduler”, and then press ENTER.
3. At the command prompt, type schtasks /delete /tn {TaskName | *} [/f] [/s computer [/u [domain\]user /p password]], and then press ENTER. For example, to delete all tasks scheduled for the local computer, type schtasks /delete /tn * /f.

Back to the top

How to Perform a Query of Scheduled Tasks

loadTOCNode(2, ’summary’);
To perform a query of scheduled tasks:

1. Click Start, click Run, type cmd, and then click OK.
2. At the command prompt, type net start, and then press ENTER to display a list of currently running services. If Task Scheduler is not displayed in the list, type net start “task scheduler”, and then press ENTER.
3. At the command prompt, type schtasks /query , and then press ENTER. Output from this example displays a table of tasks that have been scheduled to run.

For more information about how to use Schtasks.exe, search for Schtasks.exe in Windo

ws Server 2003 Help.

By: Abe Getchell 2008-10-03

This article discusses the process of recovering deleted data from an ext3 partition, on a system running Linux, using a process called data carving. This basic technique is useful in any number of situations, such as recovering data that has been accidentally deleted by a user, information removed in an attempt to erase signs of a system intrusion that could be used to track the source, or data erased by an end-user attempting to cover up an acceptable use policy infraction.

This article assumes that you have a basic understanding of ext3 and the inner workings of filesystems. It is important to note that there is a certain amount of risk associated with this process. When performed improperly, the data you are attempting to recover, or other data stored on the system, could be permanently lost. While this technique is quite accurate most of the time, and very useful in any number of different situations, it is not “forensically sound” and will not hold up legally for use in court. Special software, hardware, and procedures — or professional services — are a must in situations when legal action is required.

The tools used in this article are freely available and can be downloaded from their respective websites.

The basic recovery process

In this section we will go step-by-step through the data recovery process and describe the tools, and their options, in detail. We start by listing a directory below.

[abe@abe-laptop test]$ ls -al
total 27
drwxrwxr-x 2 abe abe 4096 2008-03-29 17:48 .
drwx—— 71 abe abe 4096 2008-03-29 17:47 ..
-rwxr–r– 1 abe abe 42736 2008-03-29 17:47 weimaraner1.jpg

In the listing above we can see that there is a file named weimaraner1.jpg in the test directory. This is a picture of my dog. I don’t want to delete it. I like my dog.

[abe@abe-laptop test]$ rm -f *

Here we can see I am deleting it. Whoops! Sorry buddy. Let’s gather some basic information about the system so we can begin the recovery process.

[abe@abe-laptop test]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 71G 14G 53G 21% /
/dev/sda1 99M 19M 76M 20% /boot
tmpfs 1007M 12K 1007M 1% /dev/shm
/dev/sdb1 887M 152M 735M 18% /media/PUBLIC

Here we see that the full path to the test directory (which is /home/abe/test) is part of the / filesystem, represented by the device file /dev/sda2.

[abe@abe-laptop test]$ su -
Password:
[root@abe-laptop ~]# debugfs /dev/sda2

Using su to gain root access, we can start the debugfs program giving it the target of /dev/sda2. The debugfs program is an interactive file system debugger that is installed by default with most common Linux distributions. This program is used to manually examine and change the state of a filesystem. In our situation, we’re going to use this program to determine the inode which stored information about the deleted file and to what block group the deleted file belonged.

debugfs 1.40.4 (31-Dec-2007)
debugfs: cd /home/abe/test
debugfs: ls -d
1835327 (12) . 65538 (4084) .. <1835328> (4072) weimaraner1.jpg

After debugfs starts, we cd into /home/abe/test and run the ls -d command. This command shows us all deleted entries in the current directory. The output shows us that we have one deleted entry and that its inode number is 1835328 — that is, the number between the angular brackets.

debugfs: imap <1835328>
Inode 1835328 is part of block group 56
located at block 1835019, offset 0×0f80

The next command we want to run is imap, giving it the inode number above so we can determine to which block group the file belonged. We see by the output that it belonged to block group 56.

debugfs: stats
[...lots of output...]
Blocks per group: 32768
[...lots of output...]
debugfs: q

Running the stats command will generate a lot of output. The only data we are interested in from this list, however, is the number of blocks per group. In this case, and most cases, it’s 32768. Now we have enough data to be able to determine the specific set of blocks in which the data resided. We’re done with debugfs now, so we type q to quit.

[root@abe-laptop ~]# dls /dev/sda2 1835008-1867775 > /media/PUBLIC/block.dat

——————–

The next thing we need to do is pull all unallocated blocks from block group 56 so we can examine their content. The dls program, from The Sleuth Kit (TSK), allows us to do just that. We simply need to know the device file, a range of blocks, and have enough space in the appropriate place to output this data. Using the information above, we can calculate the block range by multiplying the block group number and the block group size and then multiplying the block group number plus one by the blocks per group minus one. In this case, the formula would look like this:

(56 x 32768) through ((56 + 1) x 32768 – 1)

This would give us a range of 1835008 through 1867775. It’s very important that the destination of the output does not reside on the same partition as the data you’re attempting to recover. What will most likely be a large amount of data being written to disk from the output of this command could potentially overwrite the data you are trying to recover (as the blocks which stored the data from the deleted file have already been marked unallocated). You want as little disk activity as possible on the partition you’re working with. In this example, I’m using a USB thumb drive (located on /media/PUBLIC) as a location to store this data.

[root@abe-laptop ~]# mkdir /media/PUBLIC/output
[root@abe-laptop ~]# foremost -dv -t jpg -i /media/PUBLIC/block.dat -o /media/PUBLIC/output/

Next we need to attempt to extract this data from the unallocated blocks we extracted with the dls command above. To do this, we are going to use Foremost. This program is used to recover files based on header information, footer information, and internal data structures. This is the process, mentioned earlier, called data carving. First we are going to create a directory to store the foremost output (again, this should be on a separate partition). Next we are going to run the foremost command giving it the file type of jpg (which is an internally recognized type – more on custom types below), the input file, and the output directory. The output from this command is listed below.

Foremost version 1.5.3 by Jesse Kornblum, Kris Kendall, and Nick Mikus
Audit File

Foremost started at Sat Mar 29 18:02:29 2008
Invocation: foremost -dv -t jpg -i /media/PUBLIC/block.dat -o /media/PUBLIC/output/
Output directory: /media/PUBLIC/output
Configuration file: /usr/local/etc/foremost.conf
Processing: /media/PUBLIC/block.dat
|——————————————————————
File: /media/PUBLIC/block.dat
Start: Sat Mar 29 18:02:29 2008
Length: 110 MB (115941376 bytes)

Num Name (bs=512) Size File Offset Comment

0: 00033272.jpg 26 KB 17035264
1: 00033328.jpg 184 KB 17063936
2: 00033704.jpg 58 KB 17256448
3: 00033824.jpg 62 KB 17317888

[...]

*46: 00210136.jpg 2 KB 107589632
47: 00210144.jpg 3 KB 107593728
48: 00210392.jpg 6 KB 107720704
*
Finish: Sat Mar 29 18:02:29 2008

49 FILES EXTRACTED

jpg:= 49
——————————————————————

Foremost finished at Sat Mar 29 18:02:29 2008
[root@abe-laptop ~]#

As we can see, Foremost found forty-nine previously deleted jpg files (this output is also saved in a file named audit.txt in the root of the specified output directory). How do we know which is the file we are trying to recover? We could, as is most commonly done, open all of these files and see their contents. Another option is to simply compare file sizes. We know from our directory listing above that the jpg file we are looking for is 41k in size. There’s only one file that foremost extracted into the output directory that’s 41k, and indeed, 00114144.jpg is the file we are attempting to recover. Comparing size only works, of course, if you “know your data”. Integrity checking programs such as Tripwire play a big role in a recovery operation as you can identify the recovered data without ever inspecting the content, as well as verify its integrity. This becomes quite useful if the information you’re attempting to recover is confidential and you are not authorized to view the data.

Defining custom types in Foremost

As of Foremost v1.5.3, the internally supported data types that the program will recover without custom rules are jpg, gif, png, bmp, avi, exe, mpg, wav, riff, wmv, mov, pdf, ole, doc, zip, rar, htm, and cpp. If you need to recover data beyond these built-in data types, you will need to define custom types in Foremost’s configuration file (foremost.conf).

———————-

An entry that defines a type in the foremost configuration file (as explained in the documentation at the beginning of foremost.conf or in the manpage) consists of several columns: extension, case sensitivity, maximum size, header and footer (optional), and special keywords (optional). As an example that most should be familiar with, here is the entry for an html file:

htm n 50000 <html </html>

We see here that the file extension is htm (NONE can be specified if no file extension should be used during the output of extracted data), the header and footer are not case sensitive, the maximum file size is 50k bytes (which means that 50k bytes after the header will be recovered if no footer is specified or 50k bytes will be recovered if that amount of data is recovered before the defined footer is detected), the recovered file should start with “<html” (header) and end with “</html>” (footer).

The ASCII keyword can also be used when attempting to recover ASCII files. Specifying this keyword at the end of an entry will tell Foremost to extract all ASCII printable characters before and after the keyword defined. An example of this would be a type to recover a perl script. If, for example, you need to recover a perl script that you know included Crypt::CBC, you could use the following type definition:

pl y 100000 Crypt::CBC Crypt::CBC ASCII

Note that Crypt::CBC is listed in both the header and footer fields. This is done so that Foremost will recognize this as the string to search around when the ASCII keyword is used. A more general type to find perl scripts could be defined as follows:

pl n 100000 #!/usr/bin/perl #!/usr/bin/perl ASCII

When attempting to recover files that are not ASCII, hexadecimal and octal notation can be used by specifying \x[0-f][0-f] or \[0-3][0-7][0-7], respectively. Below is an example of hexadecimal notation describing the header and footers of a gif file:

gif y 155000000 \x47\x49\x46\x38\x37\x61 \x00\x3b

As you may have realized by now, Foremost is a very powerful tool. Learn its intricacies and it can be a wonderfully flexible tool in data recovery and computer security forensic operations. Read the Foremost man page or consult the configuration file for a complete guide to creating custom data types.

ext2 vs ext3 Data Recover

You may be asking yourself why this process is so much more difficult with ext3 than it is with ext2? This question is answered by one of the ext3 developers in the Linux ext3 FAQ:

Q: How can I recover (undelete) deleted files from my ext3 partition?
Actually, you can’t! This is what one of the developers, Andreas Dilger, said about it:

In order to ensure that ext3 can safely resume an unlink after a crash, it actually zeros out the block pointers in the inode, whereas ext2 just marks these blocks as unused in the block bitmaps and marks the inode as “deleted” and leaves the block pointers alone.
Your only hope is to “grep” for parts of your files that have been deleted and hope for the best.

The process, as described in this article, is the “grep” that Andreas is referring to. Hopefully, as ext3 is developed further, some effort will be put in to making this process easier and more reliable.

Conclusion

While going through this process may be necessary to recover information lost in any number of situations, it’s not a process you want to go through on a Monday morning to recover your organization’s payroll data after an administrator fat-fingers an rm command. The single most important piece of information you should take away from this article, in that vein, is to keep current, tested backups of business critical data that reside on the systems you manage. Regardless of the reason for its use, the process covered in this article is something that every system administrator and security analyst should have in their toolbelt.

Source

.SQM files are created by a number of Microsoft applications, most commonly Windows Live Messenger (previously known as MSN).

According to Microsoft, SQM files (standing for Software Quality Metrics) are used as part of their “Microsoft Customer Experience Program” and help improve their products by anonymously monitoring usage habits and reporting software errors/bugs.

To stop these files being created, you will need to disable the option in Windows Live Messenger. You can do this through the options menu:

1. Click HELP.
2. Select ‘Customer Experience Improvement Program’.
3. Tick on ‘I don’t want to participate right now’ box.
4. Click OK.

Please take note, that .SQM files are NOT viruses and do not contain spyware/malware and do not contain any personal information.

SQM files have a naming convention such as “sqmnoopt00.sqm”. They are normally found in the root folder of your hard-drive (C:) and more recently, the “Documents and settings/Application Data/Microsoft/MSN Messenger/” folder.

Dot What!? visitors have found that deleting SQM files is safe. Although probably true, we advise you to backup the files first.

################### Simple Story ###################

By default, you participate in a data-gathering program. Open Live Messenger, click on Help, then on Customer Experience Improvement Program, then de-check the radio button which says you want to participate. Try that. It will almost certainly work, and it’s not dangerous.

####################################################

Step 1: Back up important files

Red Hat has finally placed sendmail.cf in /etc/mail, where it belongs. To verify the location of your configuration file, type this command:

sendmail -d0.20 -bv | grep sendmail.cf

The default installation outputs this:

Conf file: /etc/mail/sendmail.cf (default for MTA)
Conf file: /etc/mail/sendmail.cf (selected)

Be sure to use this path when generating your new sendmail.cf from sendmail.mc, or no changes will take place. Back up your current sendmail.cf and the m4 file that generated it (probably /etc/mail/sendmail.mc):

cp /etc/mail/sendmail.cf /etc/mail/sendmail.cf~
cp /etc/mail/sendmail.mc /etc/mail/sendmail.mc~

Step 2: Make your certificate

We are also setting up STARTTLS, which allows sendmail to communicate over an encrypted layer using TLS. This is very important, as it allows us to use the LOGIN or PLAIN authentication mechanisms without transferring the password in plain text. It also allows the entire message to remain encrypted from the user’s machine to the mail server. If sendmail relays the message to another server that offers STARTTLS, the message will be encrypted again. But the most important advantage of this approach is that we get to authenticate using regular system logins and passwords, with no need to maintain a separate user database.

Red Hat’s openssl package includes a Makefile that makes it extremely easy to generate a certificate (note that on Fedora Core 4 the location is now /etc/pki/tls/certs):

cd /usr/share/ssl/certs
make sendmail.pem

Just follow the prompts and be sure to use the fully qualified domain name of the mail server for the Common Name prompt. Users will still be warned that the certificate is self-signed or not trusted, but you will prevent a warning that the certificate doesn’t match the host offering it. This certificate is suitable for testing, but you may want to investigate further about the use of certificates before deploying it in a production environment, a topic that is beyond the scope of this howto.

Step 3: Edit sendmail.mc

If you take a look at the sendmail.mc provided by Red Hat, you will notice that the necessary directives are already present but have been commented out (m4 doesn’t use the # symbol for comments, it starts a line with dnl, which stands for “delete until new line”). Since we want the easiest method possible, without sacrificing security, we need to edit these lines. Don’t cut & paste from this web page, or you may introduce unwanted characters into your configuration file that will prevent sendmail from starting.

The confAUTH_OPTIONS macro allows you to instruct sendmail not to offer plain text authentication until after a secure mechanism such as TLS is active (the p option). We are also prohibiting anonymous logins (the y option). The A option is a workaround for broken MTAs:

define(`confAUTH_OPTIONS’, `A p y’)dnl

Now we define which authentication mechanisms we will trust and use:

TRUST_AUTH_MECH(`LOGIN PLAIN’)dnl
define(`confAUTH_MECHANISMS’, `LOGIN PLAIN’)dnl

Next, we tell sendmail where to find the certificates:

define(`confCACERT_PATH’,`/usr/share/ssl/certs’)
define(`confCACERT’,`/usr/share/ssl/certs/ca-bundle.crt’)
define(`confSERVER_CERT’,`/usr/share/ssl/certs/sendmail.pem’)
define(`confSERVER_KEY’,`/usr/share/ssl/certs/sendmail.pem’)

And finally, it may be useful to increase the log level for debugging purposes (delete or comment out this line after everything is working properly):

define(`confLOG_LEVEL’, `14′)dnl

Use the m4 command to generate a new sendmail.cf:

m4 /etc/mail/sendmail.mc > /etc/mail/sendmail.cf

Be sure to use the right location for sendmail.cf, as determined earlier. Alternatively, you can use the following command in a stock Red Hat 9.0 or Fedora Core installation:

make -C /etc/mail sendmail.cf

This uses the commands in /etc/mail/Makefile to generate the new sendmail.cf configuration file.

Step 4: Test the configuration

This is where things get really interesting. sendmail must be restarted before it can use the new configuration file. Rather than simply restarting sendmail with our fingers crossed, we can test it to verify that every thing works properly. You can stop sendmail and then start it with command line options that cause it to log to a specified file. There are various ways to stop sendmail on a Red Hat/Fedora system:

service sendmail stop

or

cd /etc/mail
make stop

or

make -C /etc/mail stop

or

/etc/init.d/sendmail stop

We want to start sendmail with arguments to make it log the SMTP transaction to a special file while we are testing it:

sendmail -bD -X /tmp/test.log

Now, try to send a message from an e-mail client on another computer that does not have relay access, using your server as the outgoing mail server. You should be denied relaying. Edit your preferences so that the client uses authentication, with a login and password (not Secure Password Authentication, or SPA, which is something completely different). You should still be denied access. The last thing you need to do is to instruct the client to use SSL or TLS with the outgoing mail server (there is no need to specify a special port). After making this change, you should be able to send mail (you will be prompted to accept the certificate, however, which you might want to install to prevent further prompts). Now hit ctrl-c to stop sendmail. Restart it normally:

service sendmail restart

Now it’s time to look at the log. After the first EHLO, sendmail offers something like this:

30245 >>> 250-ENHANCEDSTATUSCODES
30245 >>> 250-PIPELINING
30245 >>> 250-8BITMIME
30245 >>> 250-SIZE
30245 >>> 250-DSN
30245 >>> 250-ETRN
30245 >>> 250-STARTTLS
30245 >>> 250-DELIVERBY
30245 >>> 250 HELP

The important thing is that AUTH is not offered here, because the channel isn’t encrypted. If you see AUTH in the first exchange, and it offers PLAIN or LOGIN, something is wrong. Look at your logs, go over the previous steps, and make sure that you generated a new sendmail.cf in the right location. The next entries in our log show that TLS is activated:

30245 <<< STARTTLS
30245 >>> 220 2.0.0 Ready to start TLS

Another EHLO takes place, followed by something like this:

30245 >>> 250-ENHANCEDSTATUSCODES
30245 >>> 250-PIPELINING
30245 >>> 250-8BITMIME
30245 >>> 250-SIZE
30245 >>> 250-DSN
30245 >>> 250-ETRN
30245 >>> 250-AUTH LOGIN PLAIN
30245 >>> 250-DELIVERBY
30245 >>> 250 HELP

Now AUTH is offered with the allowed mechanisms (but not STARTTLS, which isn’t needed here, as the channel is already encrypted). Authentication takes place, and the message is relayed to its destination.

It’s interesting to note that the username and password is Base64 encoded by the client, so it isn’t really sent as clear text:

30245 <<< AUTH PLAIN AHJvYmVydABzbHVncw==
30245 >>> 235 2.0.0 OK Authenticated

Nevertheless, it would be trivial to decode the string into the correct username/login pair (robert/slugs, in this case). Therefore, it is best to secure the transaction with TLS. If you want to verify that the transaction is encrypted, open another terminal for root, and run tcpdump:

tcpdump -s 1500 -vvxX port 25

Send a mail with easy to identify strings. You shouldn’t see your login or the message in tcpdump’s output.

Note that the certificate will be exchanged in plain text before TLS is enabled. If the mail is relayed to another server that doesn’t offer STARTTLS, you will see the content of the outgoing message in plain text.

Source

You can use below script to show the image file for public access, but public don’t have direct access to the file itself. Because its located outside of the public folder.

<?php
/* Read local file from /home/bar */
$localfile = file_get_contents("/home/userX/foo.jpg");

echo $localfile;
?>

Use below scripts :

mysql_insert_id() example