Introduction:

Synchronization between POP Clients means that all the POP Clients connected to the same POP Server for the same user have identical information. That is, the user gets the same view of mailbox irrespective of the place he is viewing it from in terms of number of messages, identity of messages & the status of each. Consider a situation where the same POP Server is used to download the mail for a particular user at a number of places.

POP Server serving the following 3 clients C1, C2 and C3          

Fig. 1

Fig. 1 depicts a typical scenario in which a user uses different machines at different places but all of which have the same POP Server. Client 1, 2 & 3 are the same users but at different locations. A user can view the mailbox from many different places, like home, office or any other common place like in Fig. 1. It would be preferred that he views the same information irrespective of the location he is viewing the mailbox from. But since the mail is downloaded at different sites though it originates from the same source, it is modified differently at different places. For example, say, a user deletes a mail from the office machine. But when he goes home, he’ll see the same mail again though it has been deleted before which is not desirable because of inconsistency. Same is the case with other status information. A message at one site may indicate that it has been replied but at the other site, it shows to have been unreplied. This results from the fact that on each machine- the mailbox, after being downloaded from the POP Server becomes local to that machine. Any changes marked on that downloaded mailbox are local and are not manifested at other places. One way to eliminate this inconsistency is that user tallies his mailbox at different sites & updates each of them comparing them all. But this is not a very good solution from user’s point of view. So this project is an attempt towards providing an application which makes sure that each of the clients is consistent without much of users’ efforts. That is, the user now gets an updated global view without any inconsistency.

What is POP?

POP stands for Post Office Protocol. It is a protocol that allows workstations to dynamically access E-mail from a mailbox server. Two commonly used meanings: “Point of Presence” and “Post Office Protocol”. A “Point of Presence” usually means a city or a location where a network can be connected to, often with dialup phone lines, so if an Internet company says they will soon have a POP in Boston, it means they will soon have a local phone number in Boston, and/or a place where leased lines can connect to their network. A second meaning, “Post Office Protocol” refers to the way E-mail software such a Pine gets mail from a mail server. It is expected that the mail will be posted from the workstation to the mailbox server via Simple Mail Transfer Protocol (SMTP).

There are three versions of POP- POP, POP2, and POP3, POP3 being the most recent & widely used. When E-mail is sent to you, it is first stored on the server until accessed by you. The protocol is for the server to be listening for connection. When a connection is opened, the server sends a greeting message and waits for commands. When commands are received the server acts on them and responds with replies.

The client opens a connection, waits for the greeting, then sends the USER and PASS commands to establish authorization to access mailboxes. Once client is authenticated, the POP is used to transmit the stored mail from the server to client’s local mailbox on his machine. When the client begins a reading transaction by giving the command for it, the server opens and locks the mailbox, and responds with the number of characters in the mailbox. Then the client issues a command to server for sending the data and the server responds to it. When all the data has been received, client sends this signal to the server. Depending upon the initial commands by the client while release of data, the server now may or may not delete the mail data from the mailbox. In any case, the server closes and unlocks the mailbox. The client terminates the session with the QUIT command.

IMAP Vs. POP

IMAP (Internet Message Access Protocol) differs from POP primarily by the fact that IMAP is specially designed to do “online” processing while POP is more suited for “offline” mail processing. In the offline mode, mail is delivered to a (usually shared) server, and a personal computer user periodically invokes a mail "client" program that connects to the server and downloads all of the pending mail to the user's own machine. In online paradigm, mail is again delivered to a shared server, but the mail client does not copy it all at once and then delete it from the server. It's more of an interactive client-server model, where the client can ask the server for headers, or the bodies of specified messages, or to search for messages meeting certain criteria. Messages in the mail repository can be marked with various status flags (e.g. "deleted" or "answered") and they stay in the repository until explicitly removed by the user. IMAP is designed to permit manipulation of remote mailboxes as if they were local. Thus, the question of inconsistency between IMAP clients does not arise as they are accessed centrally. Thus, POP Server has been specially chosen for this project because POP encounters inconsistency problems. But the advantage of POP over IMAP is that in IMAP, there is direct interaction of server with the client that leads to using lots of server resources, which could be used for other purposes otherwise. But in POP, since the client does local processing, there is minimum need of connect time and less use of server resources.

Both offline and online mailers both allow access to new incoming messages on the mail server from a variety of different client platforms. But the two paradigms reflect different requirements and styles of use and they don't mix very well. Offline works best for people who use a single client machine all the time; it is not well suited for the goals of accessing one's inbox of recent messages or saved-message folders from different machines at different times. That's because if you use offline ("download and delete") mail access from different computers at different times, your mail tends to get scattered across the different computers, unless they are all linked to a common network file system (in which case your access mode is really more online than offline.) On the other hand, the chief virtue of offline access is that it minimizes use of server resources and connect time when used via dialup.

Method:

The method used for synchronization between mail clients is – using their standard message header fields to know about their status and exchanging this information to reconcile. Each of the messages has a unique ID field called Message ID, which uniquely identifies a message in the mailbox. Its like SSN is unique to each person. Also, with each message, there is associated some status information in the form of flags which indicate the status of the message

The Message Header could look somewhat like this:

From sumit@cs.bu.edu Fri Jul 20 21:27 EDT 2001
Received: from cs.bu.edu (root@CS.BU.EDU [128.197.10.2])
by engc.bu.edu ((8.9.3.buoit.v1.0)/8.9.3/(BU-S-10/28/1999-v1.0)) with ESMTP id VAA03372
for <pkhatra@engc.bu.edu>; Fri, 20 Jul 2001 21:27:34 -0400 (EDT)
Received: from csa.bu.edu (sumit@csa [128.197.12.3])
by cs.bu.edu (8.10.1/8.10.1) with ESMTP id f6L1RXM01842
for <pkhatra@engc.bu.edu>; Fri, 20 Jul 2001 21:27:33 -0400 (EDT)
Received: from localhost (sumit@localhost)
by csa.bu.edu (8.10.1/8.10.1) with ESMTP id f6L1RTt20596
for <pkhatra@engc.bu.edu>; Fri, 20 Jul 2001 21:27:29 -0400 (EDT)
X-Authentication-Warning: csa.bu.edu: sumit owned process doing -bs
Date: Fri, 20 Jul 2001 21:27:29 -0400 (EDT)
From: Sumit Mehrotra <sumit@cs.bu.edu>
X-Sender: sumit@csa.bu.edu
To: Parminder Kaur <pkhatra@engc.bu.edu>
Subject: Re: your mail
In-Reply-To: <Pine.SOL.4.10.10107201823080.28981-100000@engc.bu.edu>
Message-ID: <Pine.SOL.4.20.0107202117480.20275-100000@csa.bu.edu>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Content-Length: 858
Status: RO
X-Status:
X-Keywords:
X-UID: 8

--Message Body—

In above Message Header, the line

Message-ID: <Pine.SOL.4.20.0107202117480.20275-100000@csa.bu.edu>

Indicates the unique identity of the message and the lines

Status: RO
X-Status:

Indicate that the message status. The Status field indicates whether the message has been viewed or not, If its value is “O”, it indicates its unseen, if its “RO”, it means the message has been viewed already. X-Status indicates the rest of status information. There can be the following five flags:

A: Message has been replied

D: Message has been marked for deletion

N: New Message (Unseen)

(Blank): Message has been seen.

*: Message has been marked as important (or any other reason), just to distinguish it from other messages

The approach that is used is: There is a client that waits for the requests by the other clients to synchronize, (hence the one which waits for request is being called server though actually it doesn’t act like a server in true sense). When another client requests the server for synchronization, the server responds by sending its Message-ID and status information (i.e. Status and X-Status flags) to the client. The client also prepares a similar file that contains Message-Ids and Status flags for each message. When the client receives similar information from the server, it compares this with its own prepared file from the mailbox and updates the mailbox using the two files. For example, suppose one mailbox (from server) shows a message M1 to be expunged while the other one shows it to be still present, the client also expunges this message to remain consistent and synchronized. Similarly, if the server says a message has been replied while the client says its not been replied still, the client also marks the message as replied. After this updation by the client, the client now knows which messages may need to be modified by the server, so it sends the Message-Ids , Status and X-Status flags from its updated mailbox to the server. When the server receives this information, it does the same thing that client did to synchronize with the server using the information obtained. So now both the client and server are synchronized.

Note: While expunging, one point may be noticed. Suppose client C1 observes that a message M is not present on client C2 that C1 has. There could be two reasons – M has been expunged or M has not been downloaded still on C2. C1 expunges this message if and only if it sees that C2 has other messages later than M. It makes sure that message M has been downloaded at least once from the server on C2 and then has been expunged, otherwise this message remains intact on C1.

Results:

To show the working of the project, following figures(pages to follow) indicate the status of mailboxes before and after the synchronization.

For example, message 28(marked seen) in Client 1 is the same as message 17(marked unseen) in Client 2. After synchronization, both of them show it to be seen. Same is the case with other status flags. Also the messages which have been expunged from one client are also expunged from the other one.

PINE 4.10 MESSAGE INDEX Folder: INBOX Message 26

+ 1 Jul 20 Sumit Mehrotra (1,908) Re: your mail
+ 2 Jul 24 Sumit Mehrotra (4,288) FW: Tobacco companies
3 Jul 29 word@m-w.com (3,924) synchronicity: M-W's Wor
4 Jul 31 word@m-w.com (3,690) aught: M-W's Word of the
+ 5 Aug 1 Ari Trachtenberg (7,574) 3.10 Can Pine be used with a
+ 6 Aug 2 Sumit Mehrotra (4,132) FW: Code Red just media
7 Aug 2 CRY Massachusetts (6,150) [cry_boston_core] Fwd:
+ 8 Aug 2 Sumit Mehrotra (2,407) Error message of the day
+ 9 Aug 2 bakhshinder25@yahoo.co (2,277) Re: Hi!!
+ 10 Aug 3 Nazi Khanna (6,787) Fwd: Free DIRECTV
11 Aug 3 word@m-w.com (3,718) point man: M-W's Word of
12 Aug 5 Amit Juneja (3,129) [cry_boston_core] Next e
13 Aug 5 Amit Juneja (4,866) [cry_boston_core] CRY WA
14 Aug 6 word@m-w.com (3,794) epigram: M-W's Word of t
15 Aug 7 Amit Juneja (1,926) Hi (fwd)
+ 16 Aug 7 William L. Oliver Jr. (5,370) GTF assignments
17 Aug 7 Joy Field (2,529) New Faculty
+ 18 Aug 7 William L. Oliver Jr. (1,338) GTF
+ 19 Aug 7 Sumit Mehrotra (2,784) Interesting
20 Aug 7 word@m-w.com (3,831) prognosticate: M-W's
21 Aug 8 Joy Field (1,698) RA Contracts
+ 22 Aug 8 Saikat Ray (1,031) health insurance
23 Aug 8 word@m-w.com (3,843) penury: M-W's Word of
24 Aug 9 Murtuza Vasowalla (9,084) [cry_boston_core] Fw:
+ A 25 Aug 9 Deshmukh, Om (2,321) tonite
N 26 Aug 9 sumitw@hotmail.com (2,831) [cry_boston_core]
D 27 Aug 9 Shivkumar B (3,127) [cry_boston_core] Spons
+ 28 Aug 9 DiscoverCard (4,302) GET A CARD THAT GIVES YO









? Help < FldrList P PrevMsg - PrevPage D Delete
O OTHER CMDS > [ViewMsg] N NextMsg Spc NextPage U Undelete

Figure 1 (Client 1 before Synchronization)

PINE 4.33 MESSAGE INDEX Folder: INBOX Message 11 of 17 NEW

+ D 1 Aug 2 India Gift House (1,763) Order No - 010716794
* 2 Aug 2 LUCKY-U (12,979) GET UP TO $200 BONUS IMM
D 3 Aug 7 Amit Juneja (2,115) Hi (fwd)
* 4 Aug 7 Joy Field (2,714) New Faculty
* 5 Aug 7 Sumit Mehrotra (2,960) Interesting
D 6 Aug 7 Ari Trachtenberg (2,023) Re: Meeting - Lab of Net
D 7 Aug 7 word@m-w.com (4,024) prognosticate: M-W's Wor
+ D 8 Aug 8 Saikat Ray (1,247) health insurance
+ D 9 Aug 8 Picture (2,674) Win $10,000! Enter Now!
D 10 Aug 8 Rajesh Munshi (23,745) [cry_boston] CRY Volunte
N 11 Aug 8 word@m-w.com (4,036) penury: M-W's Word of th
+ N 12 Aug 9 Gary (6,409) Domain Names only $8.95.
+ N 13 Aug 9 WinPrizes@inyouremail. (3,786) Play and Win!
N 14 Aug 9 Murtuza Vasowalla (9,321) [cry_boston_core] Fw:
+ 15 Aug 9 Deshmukh, Om (2,510) tonite
N 16 Aug 9 Shivkumar B (3,400) [cry_boston_core] Sponso
+ N 17 Aug 9 DiscoverCard (4,502) GET A CARD THAT GIVES YO

? Help < FldrList P PrevMsg - PrevPage D Delete
O OTHER CMDS > [ViewMsg] N NextMsg Spc NextPage U Undelete



Figure 2 (Client 2 before Synchronization)

PINE 4.10 MESSAGE INDEX Folder: INBOX Mess

D 1 Aug 7 Amit Juneja (1,926) Hi (fwd)

* 2 Aug 7 Joy Field (2,529) New Faculty

* 3 Aug 7 Sumit Mehrotra (2,784) Interesting

D 4 Aug 7 word@m-w.com (3,831) prognosticate: M-W's Wor

+ D 5 Aug 8 Saikat Ray (1,031) health insurance

6 Aug 8 word@m-w.com (3,843) penury: M-W's Word of th

7 Aug 9 Murtuza Vasowalla (9,084) [cry_boston_core] Fw:

+ A 8 Aug 9 Deshmukh, Om (2,321) tonite

D 9 Aug 9 Shivkumar B (3,127) [cry_boston_core] Sponso

+ 10 Aug 9 DiscoverCard (4,304) GET A CARD THAT GIVES YO

? Help < FldrList P PrevMsg - PrevPage D Delete

O OTHER CMDS > [ViewMsg] N NextMsg Spc NextPage U Undelete

Figure 3 (Client 1 after Synchronization)

PINE 4.33 MESSAGE INDEX Folder: INBOX Mess

D 1 Aug 7 Amit Juneja (2,115) Hi (fwd)
* 2 Aug 7 Joy Field (2,714) New Faculty
* 3 Aug 7 Sumit Mehrotra (2,960) Interesting
D 4 Aug 7 word@m-w.com (4,024) prognosticate: M-W's Wor
+ D 5 Aug 8 Saikat Ray (1,247) health insurance
6 Aug 8 word@m-w.com (4,036) penury: M-W's Word of th
7 Aug 9 Murtuza Vasowalla (9,321) [cry_boston_core] Fw:
+ A 8 Aug 9 Deshmukh, Om (2,510) tonite
D 9 Aug 9 Shivkumar B (3,400) [cry_boston_core] Sponso
+ 10 Aug 9 DiscoverCard (4,504) GET A CARD THAT GIVES YO

? Help < FldrList P PrevMsg - PrevPage D Delete
O OTHER CMDS > [ViewMsg] N NextMsg Spc NextPage U Undelete

Figure 4 (Client 2 after Synchronization)

Future Work:

This project was aimed at working with POP clients for Pine. On the same lines, this can be extended for other POP client software. The communication complexity between the two hosts can be further decreased by introducing some good communication protocol. This may ensure that the data transfer between the two POP clients is minimal and hence, the operation may be fast.