We are still actively working on the spam issue.
Setting up a Server
Need a fileserver that won't face the external internet? Check Home server.
Need to email? Set up a web server? Well, here's some advice. We're gonna try to write this page like you've never done this shit before. It does, however, assume you have at least some basic GNU/Linux knowledge. If you don't, you probably aren't ready for this. You weren't going to set up a server using Windows, were you? Jesus Christ, how horrifying.
A lot of this applies to both a physical machine as well as a VPS setup.
- 1 Common uses for a server
- 2 Home Server vs. VPS
- 3 Setting up your services
- 3.1 Centralized storage
- 3.2 Cloud storage
- 3.3 Web server
- 3.4 Setting Up Email the EASY way
- 3.5 Remote access via SSH
- 3.6 Domain and static/dynamic IP
- 3.7 Media automation
- 4 Security
- 4.1 Threat model
- 4.2 Recommended security policy
- 4.3 Mail servers
- 5 External links
Common uses for a server
- Install FTP software and run a FTP.
- Install a cloud service like Seafile to run your own Dropbox service, no privacy issues, full control, unlimited space (well, limited by how many drives you can cram in)
- Always on seedbox. Start torrents with your phone through the web interface while out, they're done by the time you're back home.
- Host a personal website
- Run your own mailserver just like Hillary!
- Warning: Running a mailserver is a shitton of work. You will get hacked all the fucking time and it's very high-maintenance.
- Run a dedicated game server
- Run various webapps, develop your own webapps
- SSH-tunnel to the server from work/school/etc to use it as a proxy, so that the admin of the network you're on can't see what sites you're going on
Home Server vs. VPS
If you want a server, you have two options: Either make your own, or rent one.
Running your own has the following benefits:
- Cheap servers are almost always VPSs, and their specs are set very low. Even bottom of the barrel hardware from 5 years ago or average hardware from 10 years ago will get you much better performance.
- Upgrading is as easy as buying a new component and sticking it in. You can install whatever software you want.
- No giving permanent access to your credit card to some company on the other side of the planet.
- You don't have to trust anyone with your data.
- When at home, you can connect to the server over LAN for blazing fast speeds.
- Can connect server to TV with HDMI for watching movies.
- Very cheap or free if you have old hardware lying around.
But renting a server also has benefits:
- Can rent the server in a country with strong privacy laws.
- Protects your identity if you use it as a proxy (assuming the company fucks you over).
- Less downtime, less maintenance problems, no electric cost.
- Probably more secure than what you'll get if you roll your own.
- No fucking around with ISP.
- At $10/mo will cost you $120 in one year. If you are buying all hardware new, a headless server (no permanent monitor or keyboard) will cost more.
A server is any machine that is on all the time, and accepts connections from the internet. Anyone who knows the IP of the server or a domain that points to that IP can try connecting. Servers can serve multiple different services, usually each service has its own port. Once it's set up you disconnect everything (monitor, keyboard, mouse) except the power cord and ethernet jack, and install something called an SSH server. You can then connect to the server from anywhere over the internet (or from inside your house over the LAN) and control it remotely.
Servers are typically administered from the command line, because GUI lags a shitton for remote access.
The first option to consider just getting a self-contained system, like a small PC designed for this purpose or a Banana board.
If you want to build your own, it can be as easy as buying some of the cheapest stuff from the Logical Increments list.
- Case: You want the smallest case that your mainboard will fit in. Unlike a desktop, you don't really need to worry about cooling or space. You can usually find some good cases like Corsair or Fractal for only $5-10 more than the cheapest one available, so that might be a good idea.
- Motherboard: Get the cheapest one you can find. Go for microATX or miniATX. The main criteria you want are:
- Compatible with a suitable CPU
- Has on-board graphics
- HDMI output is nice so you can connect to a TV
- USB 3.0 or eSATA support if you'll be using those for backing up to external drives.
- CPU: You want a cheap CPU with very low power consumption. Server CPU usage hovers around 1% and rarely goes above 5%. If your load is ever 100% it's time to monetize whatever it is you've been doing and get rich. Every extra watt is more power consumption, more heat and more noise (and with a server the noise can be a much bigger problem, depending on where it is). AMD's budget CPUs are great for these requirements.
- RAM: Any sane server OS will easily be okay with 512 MB. 1-2 GB doesn't hurt, but above 2 GB is overkill. (even 1 GB is overkill unless you're actually doing some heavy stuff)
- HDD: Anything big and cheap works fine. Even really shitty old drives can be repurposed and put in a suitable RAID, to compensate for failure, low speed or small capacity. This is probably the most critical spec of your server, besides power consumption - just stick every spare HDD you have in there.
- PSU: Your peak power consumption will probably be less than 100W, and you will never be at peak (maybe when installing OS). Unfortunately, it's hard to find decent PSUs (given that this machine is always powered, PSU is probably not a place to skimp) below 500W, so you will probably end up with those.
Ubuntu usually does retarded things with their packages and versions (lib*-ubuntu1.l2), and pulls unstable software from Debian Sid.
Rolling release distros (Arch, Fedora) are not really good for a server, because it's supposed to stay working, and it shouldn't break/change it's behavior on updates.
Gentoo is usually too much trouble to be worth it, but it's ok.
You should also consider a NAS-centric operating system for a home server - FreeBSD-based FreeNAS or NAS4Free are common choices. Both are free software and have simple GUIs to set up your services.
If you have a raspberry pi that you want to be put to use, ArkOS is a stable, Arch based distro for running a home server on a raspberry pi with a Web based GUI. ArkOS main website
A VPS is a virtual private server. When you rent a server from a company, they don't literally go and build a new machine just for you. They have huge server boxes running a VM software, and they just create a new virtual machine for you. That is your VPS.
Setting up your services
A server is perfect for this job. It is (supposedly) an always available resource on the local network. If using this in your house, you can expect reasonable speeds, even over WiFi that will let you do many daily tasks. One option is to set it up with NFS (Linux-centric, can be used on windows but it's shit) or samba, so you can watch your chinese cartoons on any device and keep your documents/whatever synchronized. This synchronization is a key benefit of the network storage.
You may want to consider a RAID array for long-term file storage. RAID is not backup, but will protect your files in case of drive failure. NAS4Free allows you to easily set up RAID arrays using UFS or ZFS.
You can store files on your server, so that they are accessible from anywhere (but password protected). These services are aimed at gaining access from the WAN (wide area network aka InterWebs). These services can be used on a LAN, but are cumbersome at times for most uses.
FTP refers to a protocol specification that stands for File Transfer Protocol. This is a protocol that allows uploading, downloading, moving, and renaming files on a remote system. This simple protocol that is supported in a large number of places but offers a reasonable amount of security to the remote system. There are not automated aspects to this, as it is just a protocol, but there are plenty of tools that work on this standard to do work for you (notably FileZilla for beginners)
a file hosting software system. Files are stored on a central server and can by synchronized with personal computers and mobile devices via the Seafile client. Files can also be accessed via the server's web interface. Seafile's functionality is similar to other popular services such as Dropbox and Google Drive, with the primary difference being that Seafile is free and open-source, enabling users to host their own Seafile servers without artificially imposed limits on storage space or client connections.
Many servers run process that have HTTP frontends (websites). OwnCloud is a similar service, with a focus on remote access to files. This service runs as a standalone process, not running on a dedicated HTTP service like apache. There are useful additions to ownCloud that do more than just server files in the web front end, such as a video player, gallery, and music player.
A web server serves up a page. The nice things about serving it from a server, than, say, Wordpress or your Dropbox share, is that now you can run web apps and server side code for a dynamic page.
Keep in mind that many server software installs (such as the email server below) involve setting up a webserver, so you may not even need to do anything.
Apache vs. nginx
The extra CPU burden of SSL is minuscule. Your server should serve up everything on HTTPS only. Keep port 80 (plain HTTP) open but redirect everything to HTTPS. If port 80 is closed, typing the address of your server into the address bar of a browser will probably fail (because the browser assumes you meant HTTP, but you have to go to HTTPS).
Issue a self-signed certificate. CAs are for jerks. Set the duration short (eg. a year) and don't forget to make a new one.
Setting Up Email the EASY way
See also: Email
Want to use your own email server to avoid the NSA? Good call! But setting up email servers can be pretty complicated. Assuming you mostly don't know what the hell you are doing, and assuming you're already secured your system per above, have a peek at iRedMail. iRedMail is an automated email and web server setup package. It works best if installed on a FRESH system - if you're already fumbled around with Apache and/or dovecot and/or postfix and failed, wipe your shit and start over with iRedMail. It will install and configure Postfix, Dovecot, Apache, and MySQL. It also installs and configures fail2ban and iptables. It includes spam filtering and greylisting. It just works. Its pretty awesome.
You will, however, still need to manually set up your DNS records (MX, SPF, and DKIM). Refer to the Email article for more on this.
If you want to get fancy and replace MySQL with MariaDB, or replace Apache with, say, Nginx, you can do that after you set up iRedMail, but any breakage is up to you to fix.
Remote access via SSH
GNU or BSD based systems
1. You usually enable the ssh server during the installation. Do this if possible, it is the simplest way.
2. If you did not setup sshd to auto start you can type:
/etc/rc.d sshd start, if you use System V init.
systemctl enable sshd.service && systemctl start sshd, if you use systemd.
3. If that does not work, you need to install
sshd with your package manager.
- Download Cygwin
- Choose "Install From Internet"
- Choose a mirror to download from
- When you get to the "Select Packages" screen, search SSH
- Choose to install the "Bin" version of the "openssh" package
- Download all dependencies and wait for setup to complete
- Open Cygwin and wait for the initial setup to complete
- Type "ssh-host-config" without the quotes
- Say yes when you get asked about privilege separation
- Say yes when asked to create a local user named sshd
- Say yes when asked to install sshd as a service
- When you see "CYGWIN=", type "ntsec tty" without the quotes (this guideexplains why)
- You should be done now setting up the SSH server. To start it, type "net start sshd" without the quotes.
If you see:
The CYGWIN sshd service is starting. The CYGWIN sshd service was started successfully.
Then everything went correctly. If not, you should Google the error.
On OS X
Simply go into sharing permissions and enable fucking everything.
- Forward port 22 to the server (Here's a guide if you don't know how)
- OPTIONAL: Assign the server an address, so you can type server.com instead of XXX.XXX.XXX.XXX (Read this)
Create a tunnel and use it as a proxy for environments that block certain DNS requests or pages and to encrypt your data
Wake on LAN
Turn on a PC on your LAN Arch Wiki guide
Host webpages, use nginx or apache debian nginx guide
You can use a proxy guide
Ziproxy (Opera style web compression, including images)
Domain and static/dynamic IP
The primary identifier of your server on the internet is the IP. Not only is it a huge pain to memorize and type the IP every time, but most likely your IP isn't static, and you get a new one from your ISP periodically.
It's a lot more convenient to have a domain like server.mydomain.com that you can use to find the server. Normally, to get a domain you need to pay, and all the good ones are already taken. But if you don't mind some extra stuff in your URL, you can use a dynamic DNS.
Make a free account, then go to subdomains and click add. Type in what subdomain you want to use, and pick one of the free domains. For example, if you want your server to be called cocks, and you pick the domain chickenkiller.com, your server will be accessible at cocks.chickenkiller.com (although this one is probably taken already). Your IP should be automatically detected, but if not, open http://ident.me/ from your server to find out what it is.
Click save, and it should pop-up in your subdomains. After a few seconds, the URL should point to your server's IP. You can test with SSH, ping, or opening in a web browser depending on what you have working. However, if your IP should change, you will need to manually update it. For doing this easily, FreeDNS actually provides an API. If you click the "ASCII" link there, you should get a table of all your domains. At the end of each line is an address. Every time you load this address in a browser, the IP will get updated.
The Python script below checks the current IP, compares to the FreeDNS's IP, and updates if they differ every time it's run. It also creates a log file in whatever directory it's in, letting you inspect past operation.
# Domains that should point to the current IP freedns_domains = set(["mysubdomain.mydomain.com"]) # FreeDNS API endpoint, accessible from https://freedns.afraid.org/api/ (use the ASCII one) freedns_check_url = "https://freedns.afraid.org/api/?action=getdyndns&sha=API_KEY_GOES_HERE" # It is assumed that this address responds with a page containing only the IP address ip_check_url = "https://ident.me/" import logging logging.basicConfig(filename="log.txt", level=logging.DEBUG, format="%(asctime)s: %(message)s", datefmt="%m/%d/%Y %I:%M:%S") logging.info("") logging.info("Script executing...") import urllib.request current_ip = urllib.request.urlopen(ip_check_url).read().decode("utf-8") logging.info("Current IP is " + current_ip) logging.info("Getting FreeDNS records...") import urllib.request records_raw = urllib.request.urlopen(freedns_check_url).read().decode("utf-8").split("\n") logging.info("Found " + str(len(records_raw)) + ".") for raw_record in records_raw: domain, current_freedns_ip, freedns_update_url = raw_record.split("|") if domain not in freedns_domains: logging.info(domain + " is not on the watch list, skipping.") else: logging.info(domain + " (" + current_freedns_ip + ") is on the watch list.") if current_ip == current_freedns_ip: logging.info("IPs already match.") else: logging.info("IPs don't match, updating...") urllib.request.urlopen(freedns_update_url) logging.info("Done.") logging.info("Script exited gracefully.")
Put this somewhere on your server, such as
~/scripts/freedns/freedns_update.py. You can test by running it manually (
python3 ~/scripts/freedns/freedns_update.py) and you can check the log (
nano ~/scripts/freedns/log.txt). To automatically run the script, set up a cron job:
crontab -e then add the following line:
0 9 * * * python3 ~/scripts/freedns/freedns_update.py
This will run the script at 9 AM every day. In principle, you could just make a cron job (
wget https://freedns.afraid.org/api/?action=getdyndns&sha=API_KEY_GOES_HERE) for each domain, but that way you wouldn't get a log.
Use a daemon like transmission or deluge
You can use a daemon like Sickbeard
You can use a daemon like Couchpotato
You can use a daemon like Headphones
Unlike a desktop, a server is always working, accepts connections from the internet (your desktop is normally firewalled and doesn't have any ports open) and is easy to discover (especially if you send mail from it). It's under a bit more risk, and its worth thinking about what intrusions you will try to prevent and how.
There are 4 main classes of attackers, grouped by what sorts of security measures are appropriate for them.
These are extremely unmotivated people like asshole flatmates or nosy neighbors (who end up on your WAN). They will rarely even try anything, and if they do, they will make the tiniest of efforts and give up at the first sign of difficulty.
Just put good passwords on your shit and don't tell them to anyone, and it will be enough for this group.
These are attackers who actually have a strong motive to get your data, but aren't competent and don't have the resources to make a serious attack. Say a burglar steals your computer. Of course he will try turning it on and seeing if there's anything interesting inside. He will not hesitate at all, unlike the previous group. Hopefully, he will see a password prompt, and after a few tries, give up on trying to guess it. However, again unlike the previous group, he will not stop there. If at all computer literate, he will try to plug the hard drive into another computer, or boot from a LiveCD. At that point it's possible that you will be the victim of identity theft.
Technically, this group has physical access, so by the common maxim of computer security, you have already lost. But in practice, they could only defeat your security theoretically. In practice, they don't know how to actually leverage that physical access, and probably won't bother trying.
Full disk encryption will deter this group, since they don't have the resources to defeat, and will simply give up and just sell the hardware at that point. Encryption takes a lot of work to defeat, and most of the shit people encrypt honestly isn't valuable or worth bothering with.
Within even the first day of your server's uptime, you will realize that there's tons people constantly trying to hack your machine. There is a sliding scale: There are tons of shitty script kiddies just randomly scanning ports over a range of IPs, and when they see your server responding, they start trying to bruteforce the SSH. This is extremely common and very easy to protect against. On the other hand, there are some extremely determined, very skilled hackers who buy 0days on the black market and use them, but these will probably be rarer, and they will prefer go after juicier targets like banks before coming after your home server full of animu porn.
The way to defend against this group is to configure your server for strong security, always keep your software up to date as new exploits appear, and keep a close eye on signs of intrusion. Defending against this group is probably the bulk of your security related maintenance workload.
At the same time, you have to recognize that you the only way to have perfect security is to turn off your server and disconnect it from the internet. Consider something like the Heartbleed bug: There is nothing you could have done to protect against it, even if you acted the moment the news broke (you could have been attacked before the news were out). That said, if a bad guy got hold of Heartbleed, he probably wouldn't attack you (there are much better targets), and just because you can't have perfect security, doesn't mean you shouldn't have any security. Lockpicks exist, but we still lock our doors.
The last class is attackers with very high competence, extensive resources, who are highly motivated to come after you. This is basically a government agency (CIA/NSA/FBI/Chinese spies) or a serious hacker (eg. hired by a company) targeting specifically you. Note that, as opposed to the 0day hacker from the previous class, these attackers are targeting specifically you. So the argument that they have "bigger fish to fry" does not apply anymore: Either the government is after you because you did something they don't like, or a someone paid a hacker to get you, either way the attacker will not stop until they've tried every trick in the book (and it's a very big book).
There isn't much you can do about this group. You could try to take some measures, but they will be very inconvenient, and some may even be dangerous or borderline illegal. For one, the government could easily gain physical access to your system by producing a warrant, and unlike a burglar, they actually could and would take advantage of that physical access. Given this, your best option is to not attract any attacks in the first place: Don't do anything illegal, don't be an important person targeted by cyber warfare, and don't piss off any hackers, or anyone who would pay a hacker. Anything beyond that is out of this document's scope.
Recommended security policy
Going by the above model, these recommendations aim to deter entirely the first two groups, deter as much of the third group as possible, without taking any additional measures for the fourth group.
- If you are behind a router, only forward ports you need.
- When installing the OS, encrypt your entire drive (except for /boot which is needed to actually boot before the disk is unlocked). Because you need to enter a password at boot to decrypt, you will not be able to reboot remotely, but the alternative of not encrypting is unacceptable. Try not to kill your server when you're not around to boot it.
- Encrypting only your home directory is not adequate. There is sensitive data outside the home directory as well.
Protecting Your Private Network
Use a DMZ, nigger.
Securing your shell
If anyone breaks into your shell, you are FUCKED. They will own your whole server within minutes. There are bots that do nothing but scan for vulnerabilities on random IP addresses all day long, and they will hammer your SSH port like it's Natalie Portman's butthole. You need to secure that shit as your first priority, and here's how:
Add a non-root user. Then, while logged in as root, do this:
Go down to the bit where it says # User privilege specification, and copy the setup for the root line. So, if your non-root user is "faggot," make it look like so:
# User privilege specification root ALL=(ALL) ALL faggot ALL=(ALL) ALL
We are going to use PuTTY in this tutorial because its consistently available on almost all platforms, including GNU/Linux (check your repos) OSX, and Windows. You can also use ssh-keygen for GNU/Linux or something else entirely, but if you already knew that, you probably wouldn't need this page, amirite?
The PuTTY suite includes a program called PuTTYGen. Run PuTTYGen to generate a public/private key pair. Go with SSH-2 DSA, 4096 bits - nobody's gonna crack that shit easily, and the NSA infiltrated the RSA, so fuck that. When done, it will display your public key. Except for the very beginning and very end, it will look like a shitton of gibberish - this is normal. Copy and paste that text into a text file, but omit the last bit that says "dsa-key-########". Put in a passphrase, this is not meant to authenticate you against the SSH server, but to keep the private key secure in case it's stolen from your client machine. Save the private key.
Open PuTTY and log in as your non-root user. Then do this:
mkdir /home/faggot/.ssh nano /home/faggot/.ssh/authorized_keys
Paste in your public key that you saved as a text file. MAKE SURE IT IS ALL ONE LINE, like so:
ssh-dsa [insanely long string of crap]
And save it.
Now take ownership of it:
chmod 600 ~/.ssh/authorized_keys
DO NOT LOG OUT OF PuTTY YET, but open a second PuTTY session and connect to your server, only this time point PuTTY to your private key file to test it out. If it all goes well, your login will look something like this:
Using username "faggot". Authenticating with public key "dsa-key-########"
Assuming that works, close your previous PuTTY session and do this:
sudo nano /etc/ssh/sshd_config
You can change the SSH port here to a random number - that's optional though, because hacker bots are gonna find it anyhow. But if you do change it don't forget to change it in PuTTY as well.
But DO make the following changes:
PermitRootLogin no PasswordAuthentication no X11Forwarding no UsePAM no
Add the following to the bottom if missing:
UseDNS no AllowUsers faggot
If you have additional users, put a space after "faggot" and add the next user, and so on if you have more.
Save these changes and restart your SSH server. On Debian, or any other system using sysvinit or OpenRC setup it would be:
sudo /etc/init.d/ssh reload
If you are using systemd, it would be:
sudo systemctl restart sshd
BOOM. Assuming all went well, you have now set up your shell so that 1) "root" cannot log in, 2) ONLY "faggot" can log in, and 3) "faggot" can ONLY log in using their private key file instead of a password. You'll still want to set up and install fail2ban or similar to secure things further.
Oh, and don't lose that private key file. You cannot recreate it, so losing it means you are doomed. Back it up in multiple places. You may wish to place a copy on a floppy drive (if you're a time-traveller from 1995) or USB stick as well, for safekeeping.
Securing your shell even more (and everything else)
Even with your shell secured, its still gonna get hammered, and that's going to get irritating quick. Given enough time, even your super-secret keyfile can eventually be guessed. Well, guess what? You can ban the SHIT out of those bots with fail2ban.
fail2ban monitors your log files (any of them, not just SSH). If it sees too many login failures, it bans the offending IP for as long as you configure it to. It won't prevent a distributed brute-force attack, but it will help a LOT.
- How To Protect SSH with fail2ban on Debian 7
- How To Protect SSH with fail2ban on CentOS 6
- How To Protect SSH with fail2ban on Ubuntu 12.04
Please note that if you are going to install iRedMail (below), it will install and configure fail2ban for you. You may still wish to tweak its settings though.
fail2ban is a utility that scans your system log files and bans anyone who tried to make logins and fails. With the default settings, 3 failed SSH logins trigger a 10 minute ban for that IP. This makes brute-forcing very difficult. Usually hackers see the ban and move on, they don't bother even waiting for the 10 minutes to run out.
sudo apt-get install fail2ban
fail2ban comes with a bunch of rules already set up. To see these, type
sudo fail2ban-client -d (fail2ban runs as root so you won't get anything without sudo). For best results, the jails should be reviewed and fine tuned. There is a manual here: http://www.fail2ban.org/wiki/index.php/MANUAL_0_8
fail2ban keeps a log where it records IPs it banned. You can see it with
nano /var/log/fail2ban.log. After a few days, a bunch of Chinese IPs should pop up.
To quickly analyze the banned IPs, run
cat /var/log/fail2ban.log | grep Ban. Paste the resulting lines into text editor of your choice and replace the regex
.*Ban with an empty string. You will now have the list of IPs. Paste these into a batch geolocator tool like this one.
Protecting from DDoS and shit
Normally, your server needs to keep a port open for each service you want to provide. However, hackers love trying to break in the moment they find an open port.
SPA solves the problem by keeping ports close when not in use. How does the server know to open them when you need to talk to it? You pick a port, say 12345, to use for SPA. The server listens on that port but never replies, so there is no way to tell that it's open (and everything else is closed). Every packet arriving on 12345 is actually inspected, the server is waiting for a packet that was encrypted with the correct key. Once a valid packet is detected, the server opens up the ports you need for a few minutes, letting you connect, and then closes them again (you stay connected).
Obviously it's bad for your mail server to get compromised. If hackers break into a mail server, they can start sending spam, so these are high value targets. As soon as they find out that your mail ports are open (and they'll find out very quickly thanks to scanning ports+harvesting your address), they'll come right for you.
If your server starts sending spam, you will be taken for a spammer by all the major spammer blacklists. Your ISP may receive abuse complaints, and they'll be wanting to ask you wtf is going on. If the attacker does something illegal (eg. spam is illegal in Murka) you could get the authorities coming after your server, too. And when all this is over and the dust settled, you'll realize that your domain is now on every blacklist under the sun, all your emails bounce so you can't talk to anyone, and it's a huge pain to get off the blacklists ("I'm totally not a spammer guys, that was just a hacker!").
Linode Library - Good beginner tutorials.