fd_set is a bitmask that doesn't have room for fds higher than 1023.

From linux sys/select.h:

typedef struct
{
__fd_mask fds_bits[__FD_SETSIZE / __NFDBITS];
  /* same as:   long fds_bits[1024 / (8 * sizeof(long))]] */
} fd_set;

That's a yes/no for fd 0 through FD_SETSIZE-1.

You can still use poll() since it uses an array and not a bitmap, but not select().

You can redefine FD_SETSIZE, but at least it used to be the case that you had to recompile all of libc then too IIRC.

I don't have access to the POSIX standard to check if this is how it's defined to always work.

Edit

This is actually documented in linux select manpage:

   An  fd_set is a fixed size buffer.  Executing FD_CLR() or FD_SET() with
   a value of fd that is negative or is equal to or larger than FD_SETSIZE
   will result in undefined behavior. 

And OpenBSD manpage (quoted here from OpenBSD 4.4):

 The default bit size of fd_set is based on the symbol
 FD_SETSIZE (currently 1024)
[...]
 However, in order to accommodate programs which might potentially
 use a larger number of open
 files with select, it is possible to increase this size within a program
 by providing a larger definition of FD_SETSIZE before the inclusion of
 <sys/types.h>.  The kernel will cope, and the userland libraries provided
 with the system are also ready for large numbers of file descriptors.

Note "userland libraries provided with the system".