Another primitive that works like a socket, but is local, is called a pipe. Unix pipes can be opened between processes running on the same machine, and once opened, they work like sockets. One process writes data into the pipe and another process reads data form the other end. It is called a pipe for two reasons. The first is that data is delivered in First-In-First-Out (FIFO) order. That is, if a single writer writes 10 bytes into a pipe, a single reader will see them in the order they were written.
The second reason for the name is that the rate that data flows in need not be the same as the rate that data flows out. Think trying to take a drink of water from a water hose. If you drink fast enough, the pipe will go dry as water is flowing in slower than you are drinking it out. If you drink too slowly, you either miss some, or the water will have to "back-up" waiting for you to drink more. Pipes, as we will study them, will be designed so that you do not have to miss any data if you are slow in draining them. We'll discuss how.
Another important features about both sockets and pipes is that they can be shared by multiple processes on the same machine. That is, you can have several processes writing into a single pipe and, at the same time, several processes reading data from the other end. Of course, they must take turns, but you can arrange that. The important thing to know, however, is that the pipe or socket agrees only to deliver the data once.
For example, if two processes are reading a pipe that has 100 bytes waiting in it, and the first reader read 25 bytes, the second reader will read from byte 26 onward. The 25 bytes that the first reader read are removed from the pipe (much like the water we discussed earlier) after the read. The same is true on the write side. If one writer writes 30 bytes and another writes 40 bytes, the writes will both go into the pipe and will not over-write each other.
In this lab, you will be implementing a restricted version of pipes that will work with pthreads. It will have some of the features of Unix pipes and sockets, but not all of them so don't panic. Our pipes will be simpler.
Notice also that the size of the data buffers that are moved around can affect the speed. If the writer writes one byte and then tells the reader that a byte is there, the overhead associated with the communication will be assessed on each byte. Let's say that is 10 ms (I don't know what the real number is, but 10 ms is probably within a factor of 10 of it). You would be "charged" 10ms for each byte moved. Now let's say that the 10 ms overhead is constant, no matter how much data is moved. The more data you move each time a reader or writer runs, the faster you will run because your "cost" per byte will be lower.
If this cost business isn't crystal clear the first time you read it, don't worry -- it will be when you are finished.
There are a couple of things about which you need to be careful. First, you must make sure that you do not over-run the size of the pipe specified in InitPipe(). For example, if the calling code calls p = InitPipe(10) and then WritePipe(p, buffer, 100) to write a buffer of 100 bytes, your code has to work even though the buffer passed into write is bigger than the total size for the pipe. To make this go, you will need to block all writers when the pipe becomes full. The full condition will be cleared when a reader successfully drains some data from the pipe. WritePipe() should not return until all of the data passed to it has been copied into the pipe, and while the pipe is full (i.e. a reader has not drained any space) WritePipe() should block.
Similarly, ReadPipe() needs to block while the pipe is empty. The empty condition clears when a writer successfully adds data to the pipe. At that time, at least one (but more is possible) reader must wake up and attempt to drain the new data.
ReadPipe() also returns a value which is the number of bytes that have actually been read. This number can be anything less than or equal to buffsize but if pipe has been closed ReadPipe() must be able to return a short buffer. For example, let's say that a writer thread runs the following code:
. . . WritePipe(p,buffer,10); ClosePipe(p); . . .and a reader thread runs
. . . bytes_read = ReadPipe(p,buffer,20); . . .Notice that ReadPipe() cannot wait until 20 bytes have been written since no more is coming (the writer has closed the pipe). In this case, ReadPipe() should fill the buffer in with the 10 bytes and return 10 as the number of bytes that were read.
Another wrinkle concerns multiple readers and writers and a closed pipe. Let's say that there are 11 reader threads and they have all called ReadPipe(p,buffer,20) but no writer has written yet. All 11 readers must be blocked because the pipe is empty. Now let's say that a writer comes along and executes WritePipe(p,buffer,10) and then ClosePipe(p). Even if you choose to deliver 1 byte to each of 10 readers (this would be legal but extremely inefficient) what should happen to the 11th? The answer is that ClosePipe() should release all blocked readers and writers. Readers should be able to keep reading until there is no more data. Any writers blocked on a full condition should be allowed to complete their writes by filling the pipe and exiting. Any reader that does not get data back should get zero and zero should indicate that the pipe is closed.
struct ThreadRec
{
int me; /* my thread number */
int buffsize;
pthread_mutex_t *Lock; /* global lock */
pthread_cond_t *Wait; /* global cond var */
int *SendRemaining; /* global counter of unsent data */
int *RecvRemaining; /* global count of unrecevd data */
int *Go; /* Go signal (to help timing) */
void *q;
};
To understand what your thread is supposed to do, it helps a great deal to
understand what it is testpipe.c is doing. Here is the
story.
testpipe takes 5 arguments. For example
testpipe nreaders nwriters buffsize quesize totalsizewhere
Once these variables are initialized, the main routine spawns nreaders copies of the routine ReaderThread() (which you have written) and nwriters copies of WriterThread(), each as a separate pthread. Each copy of ReaderThread() and WriterThread() is passed (as a void *) a pointer to its own struct ThreadRec record. The fields within this structure are initialized as follows:
First, you need to make sure you kick them off once the go code is issued. If you don't, you make have threads running "early" and that could cause problems. The easiest way to make this go is to use the global Lock and Wait fields that come into your thread. In my solution, here is the first piece of executable code in both ReaderThread() and WriterThread():
/*
* wait for the go code
*/
pthread_mutex_lock(t->Lock);
while(*(t->Go) == 0)
pthread_cond_wait(t->Wait,t->Lock);
pthread_mutex_unlock(t->Lock);
where t is a pointer to the thread structure that is passed in. Study
this code segment carefully. If it doesn't make complete sense to you, go
back an re-read the lecture notes on mutex locks and condition variables. You
will need to understand these concepts very thoroughly to make the rest of
this lab work.
Next, each thread is going to need to malloc() a buffer of size buffsize that it will use to transfer data across the pipe. WriterThreads() will use this buffer in calls to WritePipe() and ReaderThreads() will use this buffer to receive data from the pipe.
Now, here is one of the tricky parts. Your writers should each write the buffer with buffsize elements into the pipe. It doesn't really matter what data is in that buffer for timing purposes (for correctness purposes, the data will matter -- see below). What matters, however, is that all writers combined do not write more than totalsize bytes as passed to the program. The field SendRemaining points to a global integer that is shared by all writers. Each time your WriterThread() writes a buffsize chunk, it should decrement the global integer by buffsize. You must take care of the end condition as well, if buffsize does not divide totalsize exactly. If your writers write too much or two little data, it will be an error. After all totalsize bytes have been written to the pipe, your writers should exit. The last thing they have to do before exiting, though, is to call free(arg) to free the thread structure that has been passed in. If you don't free the thread structure, it is an error.
Similarly, your ReadThread()s must read until RecvRemaining has gone to zero. RecvRemaining points to a global integer and each reader should decrement that integer by the amount of data they read from the pipe. Notice that reading from a pipe may result in a buffer that is shorter than buffsize. Be careful to decrement by the bytes actually read and not just the buffer size. After all >totalsize bytes have been successfully read, the reader threads should terminate. Once all threads have terminated, testpipe will regain control, stop its timer, and calculate the speed of the transfer.
Taking care of the end condition is tricky as well. The WriterThread() that writes the last byte (as indicated by SendRemaining) should call ClosePipe() to close off the pipe. Any threads blocked inside the pipe calls for any reason (why would they be blocked?) must be released. Be sure to test the case where you have more threads than bytes to send.
#ifdef PRINT
fprintf(stdout,"thread %d writes: ",t->me);
for(i=0; i < buffsize; i++)
{
fprintf(stdout,"%d ",count);
buffer[i] = count;
count++;
}
fprintf(stdout,"\n");
#endif
where the local variable count is initialized to zero. Similarly in
your reader thread
#ifdef PRINT
fprintf(stdout,"thread: %d read: ",t->me);
for(i=0; i < buffsize; i++)
{
fprintf(stdout,"%d ",buffer[i]);
}
fprintf(stdout,"\n");
#endif
should appear just after your call to ReadPipe(). Why? Here is what
is going to happen. The makefile we will use to build your code will contain
the following targets:
testpipeprint: testpipe.c sim_pipe.o rw_threads_print.o sim_pipe.h rw_threads.h
$(CC) $(CFLAGS) -o testpipeprint testpipe.c rw_threads_print.o\
sim_pipe.o $(LIBS)
rw_threads_print.o: rw_threads.c rw_threads.h
$(CC) $(CFLAGS) -DPRINT -o rw_threads_print.o -c rw_threads.c
This is a clever (or not-so-clever) way to compile a copy of your rw_threads.c
into a binary that has the C pre-processor constant PRINT defined. The
program testpipeprint then will call the versions of your code that
have the print statements enabled. We will run small tests with the print
enabled versions of your codes to make sure that your pipe code works
correctly. Here is an example using my solution
./testpipeprint 1 1 2 3 10 thread 1 writes: 0 1 thread 1 writes: 2 3 thread: 0 read: 0 1 thread: 0 read: 2 thread 1 writes: 4 5 thread 1 writes: 6 7 thread: 0 read: 3 4 thread: 0 read: 5 thread 1 writes: 8 9 thread: 0 read: 6 7 thread: 0 read: 8 thread: 0 read: 9 bw: 0.0047 MB/s for 1 readers, 1 writers, 2 buffsize, 3 quesize 10 total size
Notice that you can see how each thread reads and writes the 10 values. All ten are written and read correctly.
./testpipe 1 1 1 1 10000 bw: 0.0279 MB/s for 1 readers, 1 writers, 1 buffsize, 1 quesize 10000 total size ./testpipe 1 1 1 10 10000 bw: 0.0657 MB/s for 1 readers, 1 writers, 1 buffsize, 10 quesize 10000 total size ./testpipe 30 30 300 20 100000 bw: 0.3812 MB/s for 30 readers, 30 writers, 300 buffsize, 20 quesize 100000 total sizeNotice that the bandwidth is different for different configurations. Why? What is the best configuration? This is for you to tell me.
Your write up is your chance to be creative. Try different machines, different configurations, and different times of day. If you do, writing a paragraph will be no problem.