NOSintro – TCP/IP over Packet Radio

An Introduction to the KA9Q Network Operating System

by Ian Wade, G3NRW

Chapter 23: FORWARDING SMTP MAIL

 

In the next three chapters we take a closer look at how to address and forward mail. There are several different methods, depending on whether you want your mail to go via the AMPRnet using SMTP, or via the AX.25 PBBS network, or a combination of both.

N.B. The techniques described in these three chapters show what is technically possible with NOS mail forwarding. This includes third-party message handling. If you are not licensed to handle third-party messages, not all of these techniques will be available to you.

 

Which Network?

The first question to consider is which network is to carry your mail. You may launch some messages into the AMPRnet, and others into the AX.25 PBBS network. This is like deciding whether to send a letter via the public Post Office network or via a private courier service.

 

Fig 23-1: Launching mail onto the AMPRnet. All stations understand SMTP.

In the ideal AMPRnet environment, you would use SMTP to forward all messages to their destinations (Fig 23-1). In this case, all forwarding stations and recipient stations understand SMTP. Think of this as the public Post Office network.

On the other hand, you could use the AX.25 PBBS network to forward all messages (Fig 23-2). This would be necessary if the recipient does not understand SMTP, and/or there is no SMTP routing available to the destination. Think of this as a private courier network.

 

Fig 23-2: If it is not possible to connect to a local SMTP station, all mail has to be launched onto the AX.25 PBBS network.

 

SMTP/PBBS Mail Gateways

In reality, of course, things aren’t as clear cut as this. Just as the Post Office makes use of private courier services, and private courier services make use of the Post Office, individual packet radio messages may in fact travel partly via SMTP routes and partly via AX.25 PBBS routes.

The network to support this (Fig 23-3) consists of SMTP/PBBS mail gateways which accept messages via SMTP and re-launch them on to the PBBS network (and vice versa). As we’ve already seen, NOS can handle both SMTP mail and AX.25 PBBS mail, and so can function as such a gateway.

Fig 23-3: SMTP/PBBS mail gateways convert message formats between AMPRnet and the PBBS network.

Launching the Mail

NOS allows you to choose how to launch mail into the combined SMTP/PBBS network. To decide which method to use, the following general rules are useful:

1. Choose a "next-door neighbour" station through which you want to forward mail (ignore for now any intermediate digipeaters or NET/ROM nodes you may need to go through to get there). Your neighbour may be a NOS station which understands SMTP, or a PBBS station which only understands AX.25.

2. If your neighbour understands SMTP, you can use SMTP to launch all of your mail. This includes mail addressed to ordinary AX.25 stations which don’t understand SMTP; somewhere along the route there will be an SMTP/PBBS mail gateway to convert messages from SMTP to PBBS format. (However, if you prefer, you can launch mail for AX.25 stations directly on to the PBBS network if you want to).

3. If your neighbour doesn’t understand SMTP, you’ll have to use AX.25 PBBS forwarding for all your mail, even for addressees running NOS.

 

General Rules for Addressing

In general, you can address mail in two ways. The examples that follow are for the built-in NOS BBS, but you will address mail in the same way if you are using an external mailer such as PCElm.

The syntax for addressing mail is as follows:

NS9BOB-5} sp user@target_mailhost
NS9BOB-5} sp
user%target
_mailhost@intermediate_mailhost

(N.B. There is no space either side of the % sign). For example:

NS9BOB-5} sp liz@ns9liz
NS9BOB-5} sp AX9TIM%BB7BBS@ns9bob

Fig 23-4: You can address mail via an intermediate mailhost if that helps message forwarding towards the target. The intermediate host translates the % character in the address to an @ character.

The target_mailhost is the eventual destination of your mail, and the user is someone (or something) that can access that mailhost to retrieve the mail (see Fig 23-4).

The intermediate_mailhost is a host somewhere along the route to the target_mailhost (again, see Fig 23-4). The intermediate_mailhost changes the first part of the address (user%target_mailhost) to read user@target_mailhost, and then sends it on towards the target.

This form of addressing, using the % symbol, is useful when you want to launch a message in a particular direction, rather than rely on automatic forwarding into unknown territory. It can save a lot of time if you know something about the network to help the message along!

Incidentally, you can use this form of addressing in a pure AX.25 PBBS network as well. This may be useful for sending messages to other countries, where some routes are known to be unreliable or where the local network has strange forwarding tables. See Fig 23-5.

Fig 23-5: Using % in addresses on the AX.25 PBBS network.

For example, SP ZZ9ZZZ%ZZ9BOX@BB7SAT lets you direct a message to the local satellite gateway BB7SAT, which then changes the address to ZZ9ZZZ@ZZ9BOX. This is probably preferable to addressing the message simply as ZZ9ZZZ@ZZ9BOX, as you then have no control over how your local network handles it, and it may finish up going via an hf AMTOR link instead. On the other hand, that may be a better bet after all ...

We’ll now look at some of the various methods of addressing mail in detail.

 

SMTP to a Known IP Address

This is the simplest method of addressing. Bob can send a message to Ken using the command:

NS9BOB-5} sp ns9ken@ns9ken

NOS discovers ns9ken’s IP address by looking it up in domain.txt:

ns9ken.ampr.org. IN A 44.199.41.2

Assuming that IP routing is set up properly, NOS connects with 44.199.41.2 and then sends the message (or, more accurately, the SMTP client in Bob’s system connects with the SMTP server in Ken’s system, and then the client and server co-operate with each other to transfer the message).

Thus for all your immediate neighbours, and for any other stations to whom you wish to send mail and for which a known route exists, you should have an IN A entry in domain.txt.

 

SMTP Client and Server

Fig 23-6 shows in more detail what happens when you send a message. Starting at the top left-hand corner, you use a mailer (such as the NOS BBS or PCElm) to compose the message.

Fig 23-6: Sending SMTP mail. The mailer places the message onto the outgoing mail queue (/spool/mqueue), then the SMTP client connects to the addressee’s SMTP server. The server places the message onto the incoming mail queue (/spool/mail), ready for reading.

The mailer places the message in the outgoing mail queue (/spool/mqueue), in two files: n.txt and n.wrk. The value n is the sequence number of the message. Thus for message number 123, the two files are called 123.txt and 123.wrk.

The sequence number itself is maintained in the file /spool/mqueue/sequence.seq.

The n.txt file contains the text of the message, in exactly the form that you composed it in the mailer, together with To:/From: address and date/timestamp information which the mailer included automatically.

The n.wrk file contains three lines. For example:

ns9ken
ns9bob@ns9bob
ns9ken@ns9ken

The first line states the target_mailhost.

The second line states who the message is from.

The third line contains the addressee.

At regular intervals (defined with the NOS smtp timer command — normally every 10 minutes), the SMTP client wakes up and looks at the outgoing message queue, by examining the n.wrk files. For each message in the queue, the client then attempts to make a connection with the SMTP server at the host listed in the first line of the n.wrk file.

In the above example, the SMTP client on the local machine attempts to connect with the SMTP server at ns9ken.

(If you don’t want to wait until the next SMTP timer interval expires, you can force the SMTP client to wake up immediately with the NOS command smtp kick).

Once contact is established, the client then transfers the message to ns9ken, where the SMTP server places it in the incoming mail directory (/spool/mail). Each user has a separate .txt file in this directory; thus a message addressed to user ns9ken@ns9ken appears in file ns9ken.txt.

Finally, the SMTP server at ns9ken outputs a message to the screen, saying that a "message for ns9ken has arrived". Ken can then log into his BBS to read it.

 

SMTP to an AX.25 Station at an SMTP Mailhost

When you want to send a message to an ordinary AX.25 station whose local mailhost is running SMTP (Fig 23-7), all you have to do is address the message to the AX.25 callsign @ the mailhost.

 

For example:

NS9BOB-5} sp AX9SAM@ns9ken

Using SMTP, the message will find its way to ns9ken, where it will be saved in Ken’s NOS BBS, in file /spool/mail/ax9sam.txt. AX9SAM can then connect to NS9KEN in the usual way, and will find the message waiting for him there.

 

Fig 23-7: Sending mail to an AX.25 station for collection at a NOS station.

SMTP via a Mail Exchanger

As already noted in Chapter 8, it’s obviously unrealistic for domain.txt to contain every known IP address in the world, and sooner or later you’ll want to send a message to a particular station whose callsign you know but whose IP address you don’t. So what do you do?

If you’re lucky, there may be a reasonably local station which has a comprehensive domain.txt file containing the address you want, and which knows how to forward your message on to its destination. In this case, you can use that station as a mail exchanger (Fig 23-8).


Fig 23-8: You can nominate a Mail Exchanger system with an IN MX record in domain.txt. Mail for ns9zzz passes to the exchanger, which then forwards it to its destination.

 

To do this, you first need to include the IP address of the gateway in your domain.txt:

ns9mxa.ampr.org. IN A 44.199.41.90

Then you add MX (Mail Exchanger) records to domain.txt for particular stations you wish to send messages to via the exchanger. For example, for messages to ns9zzz:

ns9zzz.ampr.org. IN MX 0 ns9mxa.ampr.org.

The digit 0 after IN MX is the preference value of this exchanger. A value of 0 is the highest preference.

There can be more than one MX record in domain.txt for a particular addressee, allowing you to specify alternative MX exchangers:

ns9zzz.ampr.org. IN MX 0 ns9mxa.ampr.org.
ns9zzz.ampr.org. IN MX 10 ns9mxb.ampr.org.
ns9zzz.ampr.org. IN MX 20 ns9mxc.ampr.org.

By default, NOS will attempt to forward messages for ns9zzz via the MX exchanger with the lowest preference value (i.e. ns9mxa, with preference 0). If that attempt fails, NOS will then try to forward via the exchanger with the next lowest preference (i.e. ns9mxb, with preference 10), and so on.

You can also use wild cards in MX records. This is very useful if you want to direct messages to a particular network (which may not be anything to do with AMPRnet).

For example, you can forward messages to anyone having a .com Internet address with the record in domain.txt:

*.com. IN MX 0 ns9int.ampr.org.

assuming that the MX exchanger ns9int knows how to forward on to the Internet. The wild card (*) matches any name ending in .com; e.g. messages for a.com, a.b.com and a.b.c.com all go via ns9int.

Having set up the MX records in domain.txt, you then tell NOS to use them, with the command (in autoexec.nos):

smtp usemx on

 

From now on, you can address mail via the MX exchangers with simple commands like:

NS9BOB-5} sp ns9zzz@ns9zzz
NS9BOB-5} sp bob@mwc.com

SMTP will discover that there are MX records which match ns9zzz and mwc.com, and will forward accordingly.

Fig 23-9: SMTP to an unknown IP address. If there is no entry for an addressee in domain.txt, the mail is sent to an SMTP gateway (ns9sgw), which knows how to forward it to its destination.

SMTP to an Unknown IP Address

The MX records just described are fine for forwarding mail to particular stations (or to particular networks) via specified exchangers.

More generally, however, you’ll want to send messages to people whose IP addresses you don’t know, and you have no idea how to forward these messages to them.

To handle this situation, you can nominate a default SMTP gateway to handle mail — see Fig 23-9 opposite. For example, in autoexec.nos:

smtp gateway ns9sgw

Thereafter, when you address mail to any station which does not have an IN A or IN MX entry in domain.txt, SMTP will forward the mail to the gateway. That gateway will then (hopefully!) know how to forward it onwards.

 

SMTP to an AX.25 Station at a PBBS

If you want to send a message to an ordinary AX.25 station whose local mailhost is an AX.25 PBBS, you can still launch the message using SMTP — even though neither the addressee nor his mailhost understands SMTP.

This works because there will be an SMTP/PBBS mail gateway somewhere along the route which will re-launch your SMTP message into the PBBS network (see Fig 23-10).

To address such messages, you simply use the same method as you would with an ordinary AX.25 PBBS. For example:

NS9BOB-5} sp AX9XXX@BB7BXA

NOS will look in domain.txt as usual for the IP address of the destination mailhost (BB7BXA), but won’t find it there — remember that domain.txt contains IP addresses, not AX.25 callsigns. BB7BXA is an AX.25 PBBS, so it doesn’t have an IP address.

Thus SMTP treats BB7BXA as an IP station whose address is not known, so all it can do is forward it to the default SMTP gateway station. The gateway will then either forward the message on to another SMTP gateway, or will re-launch it onto the the PBBS network. The setting up of an SMTP/PBBS gateway is described in full in Chapter 25.

 

Fig 23-10: SMTP to an AX.25 station at a PBBS. The SMTP/PBBS gateway converts the message to PBBS format then forwards it to the target PBBS.

 

The REWRITE File

There will be many times when you want to simplify the addressing of your mail, or when you want to send a particular message to more than one recipient. The NOS files /spool/rewrite and /alias are especially useful here.

The rewrite file gives us great flexibility in addressing mail, being particularly useful for handling incomplete addresses, inaccurate addresses and hierarchical addresses.

Essentially, NOS uses rewrite to map the original destination address to a new destination address. This is a one-to-one mapping, whereby the original address changes to a new address in a different format.

This is in contrast to alias expansion, where you can specify several new destinations for one original message.

Each record in rewrite is a single line containing a template for the original destination and a mapping for the corresponding replacement address — see Fig 23-11.

 

 


Fig 23-11: The file /spool/rewrite translates addresses from one format to another to suit forwarding.

 

A simple example of a rewrite record:

*@* $1%$2@ns9sss

The template for the original destination is *@*. The asterisk is a wildcard, and so the template matches any user name and mailhost; e.g. aaa@ns9aaa.

In the mapping for the replacement address ($1%$2@ns9sss), the variables $1 and $2 correspond to the first and second asterisks in the template. Thus for a message addressed to aaa@ns9aaa, $1 is aaa and $2 is ns9aaa, and so the new destination address will become aaa%ns9aaa@ns9sss.

In other words, any message originally addressed to user@mailhost will be re-addressed to user%mailhost@ns9sss.

Note that the order of the rewrite records may be important. NOS starts at the beginning of rewrite when attempting to match an address to a template, and when a match is found the scanning stops (except in the special case of a matching record containing a letter r at the end, when scanning re-starts from the beginning — this is described in more detail below).

It’s also important to note that the new destination address goes into the n.wrk file for the message. The original address (user@mailhost) remains unchanged in the associated n.txt file. In other words, rewrite changes the forwarding information in n.wrk to make it easier to launch messages in the right direction, but does not change the message itself (in n.txt) in any way.

Using the Post Office analogy again, you may write the To: address on both the letter and the envelope, but someone may change the address on the envelope afterwards to make it easier to deliver — the To: address on the letter inside the envelope remains unchanged.

 

Using wildcards for partial addresses

As well as using the asterisk wildcard to represent complete items such as user names or callsigns in the template field of the rewrite file, you can also use the asterisk to match part of an item.

For example, if you want to send all traffic addressed to German stations with callsign prefixes dj or dk via the gateway ns9deu, you could set up a rewrite record something like:

*@dj* $1%dj$2@ns9deu
*@dk* $1%dk$2@ns9deu

Then a message addressed to fritz@dj9zzz is re-addressed to fritz%dj9zzz@ns9deu, and similarly a message to hans@dk9zzz is re-addressed to hans%dk9zzz@ns9deu.

 

Rewrite File Processing

When does NOS read the rewrite file? It depends on how the message gets into the system (Fig 23-12). If you are sending a message using the NOS BBS (or if someone has logged in to your NOS BBS via telnet or an AX.25 connect), then the rewrite mapping takes place when the message goes onto the outgoing mail queue. (N.B. This only happens when you use the NOS BBS. External mailers such as PCElm are unaware of the rewrite file).

On the other hand, if the message comes into NOS from a remote system via the SMTP server, rewrite mapping takes place then.

But if a message enters the SMTP server locally, rewrite mapping does not take place; the mapping was already done when the message was put onto the outgoing mail queue.

 

Re-Scanning the rewrite File

In certain circumstances it’s useful to include several records in the rewrite file to handle incorrectly addressed mail. For example, the address fritz@dj9zzz.deu would not match any of the templates in the rewrite records listed so far. We’ve seen that the simple address fritz@dj9zzz does match one of the templates, but any other form of this address is unrecognisable.

To handle this situation, NOS provides a re-scan capability for the rewrite file. Where a rewrite record has the letter r at the end, this tells NOS that if the address matches the template in that record, perform the mapping and then go back to the beginning of the rewrite file to scan through it again.

 


Fig 23-12: The rewrite file is read either when a message arrives from a remote system via the SMTP server, or when a locally-addressed message is written to the outgoing mail queue by the NOS mailer.

 

So, to handle incorrectly addressed messages to DJ stations, the rewrite file could contain:

*@dj* $1%dj$2@ns9deu
*@dj*.deu $1@dj$2 r

The address fritz@dj9zzz.deu does not match the template in the first line, but it does match in the second line, and so the address changes to fritz@dj9zzz. Because the second line contains the letter r at the end, the scan now re-starts at the beginning of the file, and this time the new address does match the template. The address changes again, to its final form: fritz%dj9zzz@ns9deu.

Handling Incoming Bulletins

Another use for rewrite is to handle incoming bulletins (Fig 23-13).

 


Fig 23-13: The rewrite file re-addresses incoming bulletins for delivery into the mail queue.

For example, you may wish to save incoming bulletins addressed to tcpip@gbr or tcpip@eu or tcpip@www.

In this case, the rewrite mapping is simple:

tcpip@* tcpip

Thus any incoming bulletins addressed to tcpip@anything will go into your local tcpip area.

 

The Alias File

The alias file contains a look-up list which the SMTP server and external mailers use to address mail. For example, if the file contains the alias:

ken ns9ken@ns9ken

you can send a message to Ken using the BBS command sp ken, and the BBS will automatically re-address the message to ns9ken@ns9ken.

(Of course, you could have put this entry in the rewrite file instead, where it would have the same effect).

More typically, you can set up a mailing list of several recipients:

thegirls ns9pam@ns9pam ns9sue@ns9sue ns9liz@ns9liz outtray

and then simply send messages to the list members with the command sp thegirls. The last entry on the line (outtray) is a local mailbox file for saving a copy of each outgoing message.

There are three important points to keep in mind when setting up the alias file:

1.   The alias name (the first entry on each line) must be a local name. That is, it must not contain an @ character.

2.   There must be exactly one space between each field in the file.

3.   If the list of recipients is more than one line long, you can continue the list on subsequent lines. These continuation lines must start with a space or tab.

For example:

theworld ns9ccc@ns9ddd ns9eee@ns9fff ns9ggg@ns9hhh
ns9iii@ns9jjj ns9kkk@ns9lll outtray

There is a space before ns9iii at the beginning of the second line.

 

Alias File Processing

It’s interesting to see how NOS uses the alias file (see Fig 23-14).


Fig 23-14: The SMTP server uses the alias file to create multiple copies of messages.

Starting at the top left-hand corner, you can address a message to the girls with the BBS command sp thegirls. The BBS places the message as usual in the outgoing mail queue.

When the SMTP client processes the message, it finds that it is addressed to a local user (thegirls), and so the SMTP client makes an internal connection with the local SMTP server.

It is the SMTP server which reads the alias file. For each non-local recipient in the alias list (ns9pam@ns9pam, ns9sue@ns9sue and ns9liz@ns9liz), the server now places a separate copy of the original message in the outgoing mail queue. Thus when the SMTP client next wakes up it will then attempt to deliver these messages to the remote hosts.

And for each local recipient in the alias list (i.e. outtray), the server places a copy of the message in the incoming mail queue.

The important point to remember, then, is that it is the SMTP server which uses the alias file when handling mail. When you address a message to a local user name which matches an alias (like thegirls), the message leaves the outgoing mail queue under the control of the SMTP client, but immediately re-appears as incoming mail under the control of the local SMTP server.

It is only then that the server expands the alias and creates the necessary copies of the original message for forwarding.

 

Using alias to forward Incoming Messages

Because the SMTP server expands aliases for all messages, regardless of whether the messages originated locally or came from another host, you can use the alias file to forward incoming messages onwards to other hosts.

For example, if Ken addresses a message to thegirls@ns9bob, the SMTP server at ns9bob will automatically create new copies for onward transmission to all the recipients in the alias list for thegirls, plus a copy for Bob’s outtray.

 

Finding out somebody else’s aliases

If Ken sends a message to thegirls@ns9bob as just described, the end result may be totally unexpected if Ken thought that thegirls was simply a single local user at ns9bob, not an alias for several other users! Ideally Ken would like to interrogate Bob’s alias file first, to see what would happen if he subsequently addressed a message to thegirls.

When NOS is set up as described in this book, it’s not possible directly to read somebody else’s alias file (by means of FTP or a BBS download for example). This is a security precaution; the alias file is at the NOS root level (N:\), and we certainly don’t want other users snooping around with FTP or the BBS at this level.

The way around this small difficulty is for Ken to ask SMTP to do the snooping for him (Fig 23-15). He does this by giving a special telnet command:

net> telnet ns9bob 25

The number 25 is the "well-known port" number for SMTP — most of the popular protocols have a fixed port number, and 25 is always the port number for SMTP.

When Ken’s TELNET client connects with port 25 on Bob’s system, he is actually talking to the SMTP server there. The server first gives him a friendly welcome:

220 ns9bob SMTP ready

From now on Ken has to talk in language which the server understands. Fortunately SMTP-speak is very simple, with a limited vocabulary of just 9 command words: HELO NOOP MAIL QUIT RCPT HELP DATA RSET EXPN.

To find out if Bob has an alias list for thegirls, all Ken has to do now is give the EXPN (expand) command:

expn thegirls


Fig 23-15: Using telnet to well-known port 25 allows you to talk direct to the SMTP server, which you can then interrogate with the EXPN command to get the alias file.


Bob’s SMTP server will then respond with the alias file entry for
thegirls:

250-outtray
250-ns9liz@ns9liz
250-ns9sue@ns9sue
250-ns9pam@ns9pam

Bingo! Now Ken knows what will happen if he sends a message to thegirls@ns9bob!

And finally, when he has finished with Bob’s SMTP server, Ken simply gives the quit command to terminate the special TELNET session.

 

Using rewrite and alias to handle Incoming Bulletins

We’ve already seen how to set up a simple rewrite record to redirect incoming bulletins addressed to tcpip@anybody into the local tcpip area.

A reminder of the rewrite file entry:

tcpip@* tcpip

If you also wish to forward such bulletins on to Ken, you can now set up an alias entry something like:

tcpip tcpip@ns9ken tcpip

Now, when a incoming message arrives addressed to tcpip@eu, the rewrite mapping will first change the address to the local name tcpip. This now matches the alias name (the first tcpip in the alias record), so the SMTP server will place a copy of the bulletin in the outgoing mail queue for tcpip@ns9ken, and a second copy in the local tcpip area for your own system.

Note the order in which the SMTP server manipulates destination addresses: rewrite mapping takes place before alias expansion.

 

Listing the SMTP Mail Queue

From time to time you may want to see what messages are still on the outgoing mail queue awaiting forwarding. The smtp list command gives you this information:

net> smtp list
S Job Size Date Time Host From
L 123 244 09/13 07:13 ns9liz ns9bob@ns9bob ns9liz@ns9liz
274 1987 09/13 08:26 ns9tom ns9bob@ns9bob tom@ns9tom

The left-most column (S) indicates the status of the message. If there is a letter L in this column, the message is locked. That is, SMTP is in the process of forwarding it. In this case, the lock file /spool/mqueue/n.lck will exist (where n is the job number); e.g. /spool/mqueue/123.lck.

The last three columns show the contents of the work file /spool/mqueue/n.wrk.

To remove a job from the queue, use the smtp kill command. For example:

net> smtp kill 123

Back to Contents Page

Back to NOSintro Main Page.


[Copyright 1992-2009 Dowermain Ltd. All Rights Reserved. This page last modified: 21 May 2009]