MailFilter Class Reference

#include <MailFilter.h>

List of all members.

Public Types
enum	classification {   BAD_VALUE, UNKNOWN, EMAIL, SUSPECT,   GARBAGE }
Public Member Functions
	MailFilter (SpamParameters &param)
	~MailFilter ()
Private Member Functions
bool	isFromLine (const char *buf)
bool	copyToTempFiles ()
const char *	getNewTempFileName ()
FILE *	openFile (const char *fileName, const char *mode, const char *callingFunc)
void	closeFile (FILE *fp, const char *fileName, const char *callingFunc)
bool	writeLine (const char *buf, FILE *fp, const char *fileName, const char *callingFunc)
const char *	readLine (char *buf, const size_t bufSize, FILE *fp, const char *fileName, const char *callingFunc)
void	append_file (const char *srcfile, const char *destfile)
void	error_append_file (const char *srcfile, const char *destfile)
classification	checkMail (const char *tempFileName, SpamParameters &params, HeaderInfo &headInfo)
Private Attributes
size_t	mFileCount
Logger	log
std::vector< char * >	fileNames

---

Detailed Description

The MailFilter class constructor is the entry point for email processing. The constructor calls MailFilter support routines that read the email from stdin into one or more temporary files (if there is more than one email). The checkMail function is called to analyze the email. Then MailFilter functions are called to append the email to the "inbox" file, the junk_mail file, the garbage_mail file or discard the email (if it is marked as garbage).

Definition at line 60 of file MailFilter.h.

---

Constructor & Destructor Documentation

MailFilter::MailFilter (  SpamParameters &  params  )

Filter email read from stdin. Email is read from stdin. If there is more than one email, each email will be placed in a unique temporary file. Each file is then processed in an attempt to determine if it is valid email or spam. Email is catagorized as EMAIL (or UNKNOWN), SUSPECT, or GARBAGE. Email is marked as garbage when a "kill_word" is found. If the "kill_base64" flag is included in the SpamFilterParams file, email that contains base64 encoded data will be marked as garbage. Email that is marked as garbage is only placed in the garbage file when the SpamFilterParams flag "keep_garbage" is included. Otherwise email marked as garbage is discarded. This keeps the number of headers that must be reviewed in the junk_mail file as low as possible. Parameters: params  a reference to a SpamParameters object containing information from the SpamFilterParams file. Definition at line 513 of file MailFilter.C. References append_file(), checkMail(), copyToTempFiles(), error_append_file(), Logger::errorFound(), fileNames, Logger::getLogger(), SpamParameters::hasFlag(), Logger::log(), log, mFileCount, and HeaderInfo::subject(). { // file for email const char* INBOX = "inbox"; // fiile for email that is suspected of being spam const char* SPAM = "junk_mail"; // File for email that is "garbage". const char* GARBAGE_MAIL = "garbage_mail"; mFileCount = 0; log = pLogger->getLogger("MailFilter"); log.log(Logger::DEBUG, "MailFilter", "enter"); bool doGarbageTrace = params.hasFlag("trace_garbage") && (! params.hasFlag("keep_garbage")); // read mail file from stdin into one or more temporary file if (copyToTempFiles()) { size_t numFiles = fileNames.size(); for (int i = 0; i < numFiles; i++) { const char *tempFileName = fileNames[i]; HeaderInfo headInfo( doGarbageTrace ); char msg[256]; classification kind = checkMail(tempFileName, params, headInfo); Logger::LogLevel mode; switch (kind) { case UNKNOWN: { // If the email is classified as "UNKNOWN" then something is // wrong. But we don't want to lose the email, so append it // to the inbox. sprintf(msg, "email classified as UNKNOWN"); append_file( tempFileName, INBOX ); mode = Logger::ERROR; } break; case EMAIL: { sprintf(msg, "Subject: %s added to mail in %s", headInfo.subject(), INBOX ); append_file( tempFileName, INBOX ); mode = Logger::DEBUG; } break; case SUSPECT: { sprintf(msg, "Subject: %s added to suspected spam in %s", headInfo.subject(), SPAM ); append_file( tempFileName, SPAM ); mode = Logger::DEBUG; } break; case GARBAGE: { if (params.hasFlag("keep_garbage")) { sprintf(msg, "Subject: %s is garbage, copied to %s", headInfo.subject(), GARBAGE_MAIL ); append_file( tempFileName, GARBAGE_MAIL ); } else { sprintf(msg, "Subject: %s deleted", headInfo.subject() ); } mode = Logger::DEBUG; } break; case BAD_VALUE: { // something went wrong processing the e-mail sprintf(msg, "Mail filter error: Subject = %s", headInfo.subject() ); // Append it to the inbox so it is not lost. The error_append_file // function will add a marker to the file to indicate that there // was an error error_append_file( tempFileName, INBOX ); mode = Logger::ERROR; } break; default: { sprintf(msg, "bad classification value" ); mode = Logger::ERROR; } break; } // switch log.log( mode, "MailFilter", msg ); if (! log.errorFound()) { // remove temporary file sprintf(msg, "removing %s", tempFileName ); log.log(Logger::DEBUG, "MailFilter", msg ); int unlinkRslt = unlink( tempFileName ); if (unlinkRslt != 0) { sprintf(msg, "error unlinking %s. Error = %s\n", tempFileName, strerror(errno)); log.log(Logger::ERROR, "MailFilter", msg ); } } else { sprintf(msg, "email that caused the error is in %s", tempFileName ); log.log(Logger::ERROR, "MailFilter", msg ); } } // for } // if copyToTempFiles log.log(Logger::DEBUG, "MailFilter", "exit"); } // MailFilter constructor

MailFilter::~MailFilter (   )

Recover the memory allocated for the temporary file name in fileNames. Definition at line 166 of file MailFilter.C. References fileNames. { size_t numFiles = fileNames.size(); for (int i = 0; i < numFiles; i++) { char *pStr = fileNames[i]; delete [] pStr; } } // ~MailFilter

---

Member Function Documentation

void MailFilter::append_file (  const char *  srcfile, const char *  destfile )  [private]

append_file Append srcfile to destfile. This is used when the destination of the email is decided. The email will either be appended to the junk file or back to the email box. A carriage return is added between the e-mails. This avoids having e-mails run together. Definition at line 397 of file MailFilter.C. References Logger::log(), and log. Referenced by MailFilter(). { char msgbuf[ 128]; const char *read_only = "r"; const char *append = "a+"; FILE *read_fp; FILE *write_fp; log.log(Logger::DEBUG, "append_file", "enter"); if ((read_fp = fopen( srcfile, read_only )) != NULL) { if ((write_fp = fopen( destfile, append )) != NULL) { char buf[ 4096 ]; size_t amt_read; size_t amt_written; fprintf(write_fp, "\n"); // add a carriage return (blank line) while ((amt_read = fread(buf, 1, sizeof(buf), read_fp)) > 0) { amt_written = fwrite(buf, 1, amt_read, write_fp ); if (amt_written < amt_read) { char *err_reason = strerror( errno ); sprintf(msgbuf, "error writing file %s. Reason = %s", destfile, err_reason); log.log(Logger::ERROR, "append_file", msgbuf ); } } // while fclose( write_fp ); } else { char *err_reason = strerror( errno ); sprintf(msgbuf, "append_file: error opening file %s. Reason = %s", destfile, err_reason ); log.log(Logger::ERROR, "append_file", msgbuf ); } fclose( read_fp ); } else { char *err_reason = strerror( errno ); sprintf( msgbuf, "append_file: error opening file %s. Reason = %s", srcfile, err_reason ); log.log(Logger::ERROR, "append_file", msgbuf ); } log.log(Logger::DEBUG, "append_file", "exit"); } // append_file

MailFilter::classification MailFilter::checkMail (  const char *  tempFileName, SpamParameters &  params, HeaderInfo &  headInfo )  [private]

Attempt to determine of the email is valid or if it is spam. The email header is checked first. After checking the email header, if it is still unknown whether the email is valid or spam, check the email body. If the email is not found to be "guilty" (e.g., spam) it is assumed to be innocent (valid email). Parameters: tempFileName  the name of the file containing the email to be checked params  a reference to the SpamParameters object containing information from the SpamFilterParams file. headInfo  a reference to a HeaderInfo object which is used to encapsulate information about the email header. The HeaderInfo object is used to generate an garbage_trace entry (if garbage tracing is turned on and the email is discarded). HeaderInfo is also used to generate debug log messages. Definition at line 466 of file MailFilter.C. References MailBody::checkBody(), MailHeader::checkHeader(), MailHeader::getBoundaryStr(), HeaderInfo::klass(), Logger::log(), and log. Referenced by MailFilter(). { const char *mode = "r"; classification mailClass = EMAIL; char msgbuf[256]; log.log(Logger::DEBUG, "checkMail", "enter"); FILE *fp = openFile(tempFileName, mode, "checkMail"); if (fp != NULL) { MailHeader headFilter( params, headInfo ); mailClass = headFilter.checkHeader(fp); if (mailClass == UNKNOWN) { MailBody bodyFilter( params, headInfo ); const char *boundaryStr = headFilter.getBoundaryStr(); mailClass = bodyFilter.checkBody(boundaryStr, fp); headInfo.klass(mailClass); } fclose( fp ); } log.log(Logger::DEBUG, "checkMail", "exit"); return mailClass; } // checkMail

bool MailFilter::copyToTempFiles (   )  [private]

copyToTempFiles Read one or more emails from stdin. Each email will be copied into a temporary file, whose name will be inserted into the fileNames vector. When I started writing this software, I thought that it was one email per invocation of the mail filter. While testing the mail filter, I found that more than one email may arrive at one time. I don't know why this is. It could be the way mail is handled by my ISP. It could be that so much spam is sent out that SPAM clumps together. Or it could be that spammers include two emails in one mail transaction. What ever the case, this function will separate each email into its own temporary file. Mail tools find the start of an email via the "From" line. The format for this line is: From <address> <date> The date is in UNIX/Linux ctime(3) format. If the date is not included mail software cannot find the start of the email. The same technique is used here. Definition at line 285 of file MailFilter.C. References getNewTempFileName(), isFromLine(), Logger::log(), log, readLine(), and writeLine(). Referenced by MailFilter(). { const char *mode = "w"; char buf[ 1024 ]; char msgbuf[ 128]; bool copyOK = true; const char *inLine = 0; const char *fileName = 0; FILE *fp = 0; bool firstFrom = true; log.log(Logger::DEBUG, "copyToTempFiles", "enter"); do { if ((inLine = readLine(buf, sizeof(buf), stdin, 0, "copyToTempFiles"))) { if (isFromLine( buf )) { if (firstFrom) { firstFrom = false; } else { closeFile( fp, fileName, "copyToTempFiles"); } fileName = getNewTempFileName(); fp = openFile( fileName, mode, "copyToTempFiles" ); } // isFromLine if (fp) { if (! writeLine(buf, fp, fileName, "copyToTempFiles")) { copyOK = false; break; } } } // inLine = readLine else { if (! feof(stdin)) { copyOK = false; } else { closeFile( fp, fileName, "copyToTempFiles"); } } } while (inLine != 0); log.log(Logger::DEBUG, "copyToTempFiles", "exit"); return copyOK; } // copyToTempFiles

void MailFilter::error_append_file (  const char *  srcfile, const char *  destfile )  [private]

Append the email to the "inbox" file and include an error line in the email header. Something has gone wrong. The email should not be lost, so it is appended to the "inbox". An error line is included in the header. Definition at line 341 of file MailFilter.C. References Logger::log(), and log. Referenced by MailFilter(). { static const char *SUBJECT = "subject"; static size_t SUBJECT_LEN = strlen( SUBJECT ); char msgbuf[ 128]; const char *read_only = "r"; const char *append = "a+"; FILE *read_fp; FILE *write_fp; log.log(Logger::DEBUG, "error_append_file", "enter"); if ((read_fp = fopen( srcfile, read_only )) != NULL) { if ((write_fp = fopen( destfile, append )) != NULL) { char line[ 4096 ]; size_t amt_read; size_t amt_written; fprintf(write_fp, "\n"); // add a carriage return (blank line) while (fgets(line, sizeof(line), read_fp) != 0) { fputs(line, write_fp); // append "X-MailFilterError:" after the "Subject:" line if (SpamUtil().match(line, SUBJECT_LEN, SUBJECT)) { fprintf(write_fp, "X-MailFilterError:\n"); } } // while fclose( write_fp ); } else { sprintf(msgbuf, "error opening file %s", destfile ); log.log(Logger::ERROR, "error_append_file", msgbuf ); } fclose( read_fp ); } else { sprintf( msgbuf, "error opening file %s", srcfile ); log.log(Logger::ERROR, "error_append_file", msgbuf ); } log.log(Logger::DEBUG, "error_append_file", "exit"); } // error_append_file

const char * MailFilter::getNewTempFileName (   )  [private]

Create a new file name and enter it into the fileNames vector. Definition at line 50 of file MailFilter.C. References fileNames, and mFileCount. Referenced by copyToTempFiles(). { const char* TEMP_NAME_ROOT = "mail_temp"; const size_t BUF_SIZE = 64;; char *pBuf = new char[ BUF_SIZE ]; int pid = getpid(); // create a unique temporary file name mFileCount++; sprintf(pBuf, "%s_%d_%d", TEMP_NAME_ROOT, pid, mFileCount ); fileNames.push_back( pBuf ); return pBuf; } // getNewTempFileName

bool MailFilter::isFromLine (  const char *  buf  )  [private]

Check to see if the line in buf is a From line that starts an email. The start of an email is recognized by the leading From line. This line has the format: From some@address.com <date> Where <date> is <dow> <mon> dd hh:mm:ss yyyy (UNIX/Linux ctime(3) format) <dow> = a three letter day of the week: Sun, Mon, Tue, Wed, Thu, Fri, Sat <mon> = a three letter month: Jan, Feb, Mar, Apr, May, Jun, Jul, Aug,... dd = numeric day of the month (e.g., 1..31) hh = hour (1..24) mm = minute ss = second yyyy = year This function looks for "From ", followed by a date. A date is recognized by looking for a valid day of the week, a valid month and a valid day of the month. While is is till possible that another line in an email will be accidently recognized, it is very unlikely. Definition at line 206 of file MailFilter.C. References Logger::log(), and log. Referenced by copyToTempFiles(). { static const char *FROM = "From "; static const size_t FROM_LEN = strlen( FROM ); static const char *dow[] = {"Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat", 0 }; static const char *mon[] = {"Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec", 0 }; char msgbuf[128]; bool isFrom = false; if (buf != 0) { // find "From " if (strncmp(buf, FROM, FROM_LEN) == 0) { sprintf(msgbuf, "Found from line: %s", buf ); log.log(Logger::DEBUG, "isFromLine", msgbuf); // find a day-of-the-week const char *pDOW = 0; for (const char **pDay = dow; *pDay != 0; pDay++) { if ((pDOW = strstr(buf, *pDay)) != 0) { break; } } // for if (pDOW != 0) {// if a day-of-the-week was found, find a month const char *pStartMonth = 0; for (const char **pMon = mon; *pMon != 0; pMon++) { if ((pStartMonth = strstr(pDOW+3, *pMon)) != 0) { break; } } // for // look for the day of the month (1..31) if (pStartMonth != 0) { const char *pDate = pStartMonth + 3; pDate = SpamUtil().skipWhiteSpace( pDate ); if (isdigit(*pDate)) { int date = atoi( pDate ); if (date >= 1 && date <= 31) { log.log(Logger::DEBUG, "isFromLine", "Found start of email"); isFrom = true; } } } } // if pDow != 0 } } return isFrom; } // isFromLine

const char * MailFilter::readLine (  char *  buf, const size_t  bufSize, FILE *  fp, const char *  fileName, const char *  callingFunc )  [private]

Read a line of text. Print an error message to the log file if there is an error. Definition at line 137 of file MailFilter.C. References Logger::log(), and log. Referenced by copyToTempFiles(). { char *inLine = 0; *buf = '\0'; if ((inLine = fgets( buf, bufSize, fp )) == 0) { if (! feof(fp)) { char msgbuf[128]; char *err_reason = strerror( errno ); if (fileName != 0) { sprintf(msgbuf, "Error reading from %s. Reason = %s", fileName, err_reason ); } else if (fp == stdin) { sprintf(msgbuf, "Error reading from stdin. Reason = %s", err_reason ); } log.log(Logger::ERROR, callingFunc, msgbuf ); } } return inLine; } // readLine

bool MailFilter::writeLine (  const char *  buf, FILE *  fp, const char *  fileName, const char *  callingFunc )  [private]

Write a line of text. Print a log message to the log file if there is an error. Note that the fgets result is compared to EOF, rather than zero. This is necessary for portability, since apparently zero is not necessarily returned on success. Definition at line 115 of file MailFilter.C. References Logger::log(), and log. Referenced by copyToTempFiles(). { bool writeOK = true; if (fputs( buf, fp ) == EOF) { writeOK = false; char msgbuf[128]; char *err_reason = strerror( errno ); sprintf(msgbuf, "Error writing to %s. Reason = %s", fileName, err_reason ); log.log(Logger::ERROR, callingFunc, msgbuf ); } return writeOK; } // writeLine

---

Member Data Documentation

std::vector<char *> MailFilter::fileNames [private]

The names of the temporary files created for the emails read from stdin Definition at line 72 of file MailFilter.h. Referenced by getNewTempFileName(), MailFilter(), and ~MailFilter().

Logger MailFilter::log [private]

Logger object Definition at line 70 of file MailFilter.h. Referenced by append_file(), checkMail(), copyToTempFiles(), error_append_file(), isFromLine(), MailFilter(), readLine(), and writeLine().

size_t MailFilter::mFileCount [private]

A count for the temporary files created for the emails read from stdin Definition at line 68 of file MailFilter.h. Referenced by getNewTempFileName(), and MailFilter().

---

The documentation for this class was generated from the following files:

• MailFilter.h
• MailFilter.C

---