Set up rssh with chroot jail on AIX 5.3

datePosted on 16:06, March 21st, 2008 by yk

Scenario: Your AIX server has OpenSSH installed and sFTP configured. How do you restrict sFTP users from accessing other shells (like ksh, bash, csh, scp, ftp, etc)? How do you prevent them from traversing through your directories and potentially doing all sorts of damage?

The answer: Set up rssh with chroot jail. Restricted shell (rssh) is a replacement for the standard shell (in AIX’s case, ksh) to be used with OpenSSH that lets you restrict shell access to only sftp and scp. It supports chroot jail which is a method to sandbox users and programs in a specific directory.

How do you set it up? Russell Martin wrote an excellent article with detailed explanations and instructions on the subject. I recommend that you read the article to understand what we’re trying to achieve.

Ok, why am I writing this post then? Well, even with the abundance of information from Russell’s article, it still took me a long time to set it up on an AIX 5.3ML6 server running OpenSSH 4.3p2. So, I’m writing down everything that I’ve done just in case someone got stuck like me.

(1) Installing rssh

  1. If you do not have gcc or any other compiler installed on your system, head on to IBM AIX Toolbox for Linux Applications download page and grab the gcc 4.0.0(5.3) RPM. To install the gcc rpm, just copy it into your server and issue the command:
    rpm --install gcc-4.0.0-1.aix5.3.ppc.rpm
  2. Download rssh 2.3.2 source tarball from and copy it into a temporary directory in your server.
  3. Unzip and untar the archive:
    gunzip -d rssh-2.3.2.tar.gz
    tar xvf rssh-2.3.2.tar
  4. The included INSTALL file provides instructions to configure, compile and install rssh. However, there isn’t a GNU libc compatible malloc present in AIX 5.3 and this will cause the configuration script to configure the MakeFile errornously (read about this problem here), so you will have to issue the following command before attempting to follow the instructions:
    export ac_cv_func_malloc_0_nonnull=yes Now we can start to run the commands:
    su root
    make install
    However, in my case sftp-server is located in the /usr/sbin directory, so for the first command, I have to do this instead:
    ./configure --with-sftp-server=/usr/sbin/sftp-server

(2) Creating an user and chroot jail

  1. First of all, I’ll explain how I’ll be configuring my chroot jail. In my environment, I’ll be creating a special user called sftpuser for sFTP access. I will “jail” the user in the /var/file_transfer directory and the home directory for sftpuser will be /var/file_transfer/sftpuser.So in effect, when sftpuser logs in, he will land in the /var/file_transfer/sftpuser directory of the server but it will appear as /sftpuser directory to him. If he issues a cd / command, he will arrive in the /var/file_transfer directory of the server, but to him, he is already in the root directory.
  2. The first step is to create the user in the system. I’m using smit, so I’ll explain using that method. Type in:
    smit user
    Select the option Add a User.
    Key in sftpuser as the User NAME.
    Key in /var/file_transfer/sftpuser as the HOME directory.
    Key in /usr/bin/ksh as the Initial PROGRAM for now.
    Then, proceed and create the user.
  3. After creating the user, we need to set a password for it. Type in:
    smit user
    Select the option Change a User’s Password and set a password for it. After that, try to log in using telnet or SSH and you will be prompted to change the user password.
  4. Now, we have to set the user’s shell to rssh so that the user will be controlled by it. Type in:
    smit user
    Select the option Change / Show Characteristics of a User
    Key in sftpuser as the User NAME and proceed.
    Key in /usr/bin/rssh as the Initial PROGRAM and proceed to save the changes. Now, we’re done creating the user in our system.
  5. To set up the chroot jail, I modified the script downloaded from Russell’s article to cater for AIX 5.3. You can download the script here. Upload this to any directory on your server and chmod to make it executable:
    chmod 755 chroot.shBefore you run the script create the chroot directory first:
    cd /var
    mkdir file_transfer
    Then, edit the paths in the script:
    CHROOTPATH=/var/file_transfer #defines the path of your chroot directory (the directory where you want to confine users to).
    USER_HOME_DIR=sftpuser #defines the path of the user's home relative to the CHROOTPATH
    USER_NAME=sftpuser #defines the user name
    GROUP_NAME=staff #defines the user group
    After changing the paths, run the script:
  6. The next step is to edit the rssh configuration file (mine resides is /etc/rssh/rssh.cfg). The comments in the configuration file is pretty self-explanatory. The resulting file should have the following uncommented lines:
    logfacility = LOG_USER
    umask = 022
    chrootpath = "/var/file_transfer"

Congratulations! You have finished setting up rssh with chroot jail. Now you can sFTP in and see if all works fine.

In you run into any problems like the dreadful Connection Closed, you will have to check the syslog to see what the problem is.

categoryPosted in Tips and Tricks | printPrint

2 Responses to “Set up rssh with chroot jail on AIX 5.3”

  1. Joe Lewis on April 22nd, 2010 at 6:40 pm

    I have read your documentation carefully as you stated and I am still getting the error: fatal error identifying the correct command (This should never happen). any clue what is going on here? Thanks, Joe

  2. yk on April 22nd, 2010 at 7:16 pm

    Hi Joe, what firmware are you running now?

    At which step did the error occur?

Leave a Reply

Name: (required)
Email: (required) (will not be published)
Spam protection: