We are still actively working on the spam issue.

Difference between revisions of "Setting up a Server"

From InstallGentoo Wiki
Jump to: navigation, search
(Security)
(Web server: server error? Failed change fixed)
 
(17 intermediate revisions by 3 users not shown)
Line 5: Line 5:
 
A lot of this applies to both a physical machine as well as a [[VPS]] setup.
 
A lot of this applies to both a physical machine as well as a [[VPS]] setup.
  
= Common uses for a server=
+
=== Common uses for a server===
* Install FTP software and run a FTP.
+
* Install FTP software and run an FTP site.
 
* Install a cloud service like Seafile to run your own Dropbox service, no privacy issues, full control, unlimited space (well, limited by how many drives you can cram in).
 
* Install a cloud service like Seafile to run your own Dropbox service, no privacy issues, full control, unlimited space (well, limited by how many drives you can cram in).
 
* Always on seedbox. Start torrents with your phone through the web interface while out, they're done by the time you're back home.
 
* Always on seedbox. Start torrents with your phone through the web interface while out, they're done by the time you're back home.
Line 17: Line 17:
 
* Run a VPN for location spoofing or security when you're out and about.
 
* Run a VPN for location spoofing or security when you're out and about.
  
= Home Server vs. VPS =
+
= Getting Started =
If you want a server, you have two options: Either make your own, or rent one.
+
[[Setting up a Server/Home or Remote?|Home or Remote?]]
  
Running your own has the following benefits:
+
[[Setting up a Server/Choosing an Operating System|Choosing an Operating System]]
* Cheap servers are almost always VPSs, and their specs are set very low. Even bottom of the barrel hardware from 5 years ago or average hardware from 10 years ago will get you much better performance.
+
=Setting up your services=
* Upgrading is as easy as buying a new component and sticking it in. You can install whatever software you want.
+
Most packages have clear tutorials on their repo/project site. Here are some handpicked guides as well as some written by other anons.
* No giving permanent access to your credit card to some company on the other side of the planet.
+
*[[Setting up a Server/Mail|Mail]]
* You don't have to trust anyone with your data.
+
*[[Setting up a Server/DNS|DNS]]
* When at home, you can connect to the server over LAN for blazing fast speeds.
+
*FTP
* Can connect server to TV with HDMI for watching movies.
+
*Cloud Storage
* Very cheap or free if you have old hardware lying around.
+
*Web Server
 
+
*ZNC
But renting a server also has benefits:
+
*SSH
* Can rent the server in a country with strong privacy laws.
+
*Everything in Docker
* Protects your identity if you use it as a proxy (assuming the company doesn't fucks you over).
+
*HAProxy (for multiple machines behind one ip).
* Less downtime, less maintenance problems, no electric cost.
+
*VPN
* Probably more secure than what you'll get if you roll your own.
+
*Media Automation (Sickbeard, Flexget etc.)
* No fucking around with ISP.
+
*Security
* At $10/mo will cost you $120 in one year. If you are buying all hardware new, a headless server (no permanent monitor or keyboard) will cost more.
+
*Media Streaming with icecast/ffmpeg/cvlc
 
 
== Home server ==
 
A server is any machine that is on all the time, and accepts connections from the internet. Anyone who knows the IP of the server or a domain that points to that IP can try connecting. Servers can serve multiple different services, usually each service has its own port. Once it's set up you disconnect everything (monitor, keyboard, mouse) except the power cord and ethernet jack, and install something called an SSH server. You can then connect to the server from anywhere over the internet (or from inside your house over the LAN) and control it remotely.
 
  
Servers are typically administered from the command line, because GUI lags a shitton for remote access.
+
=Recommended software=
 +
*Mail: see [[Setting up a Server/Mail]]
 +
*DNS: see [[Setting up a Server/DNS#Authoritative]]
 +
*FTP: vsftpd, glftpd (nonfree)
 +
*IRCd: inspircd, ngircd, kike
 +
*IRC Bouncer: znc
 +
*XDCC: iroffer-dinoex
 +
*XMPP: Prosody
 +
*Proxy: danted
 +
*Usenet Server: Leafnode, InterNetNews
 +
*Web Cache/Reverse Proxy: varnish, squid, nginx
 +
*Web: nginx, darkhttpd (use stunnel for ssl)
 +
*Config Management: cdist, ansible, salt
 +
*Control Panel: Webmin, Cockpit
 +
*Direct Connect Server (DCC): uhub, luadch
 +
*Monitoring:
  
=== Hardware===
+
==Less common ==
The first option to consider just getting a self-contained system, like a small PC designed for this purpose or a [[Single board computers|Single board computer]].
+
*BBS Server: Mystic BBS (BBSs are dead)
 +
*MUD: PennMUSH
  
If you want to build your own, it can be as easy as buying some of the cheapest stuff from the Logical Increments list.
 
  
* Case: You want the smallest case that your mainboard will fit in. Unlike a desktop, you don't really need to worry about cooling or space. You can usually find some good cases like Corsair or Fractal for only $5-10 more than the cheapest one available, so that might be a good idea.
 
* Motherboard: Get the cheapest one you can find. Go for microATX or miniATX. The main criteria you want are:
 
** Compatible with a suitable CPU
 
** Has on-board graphics
 
** HDMI output is nice so you can connect to a TV
 
** USB 3.0 or eSATA support if you'll be using those for backing up to external drives.
 
* CPU: You want a cheap CPU with very low power consumption. Server CPU usage hovers around 1% and rarely goes above 5%. If your load is ever 100% it's time to monetize whatever it is you've been doing and get rich. Every extra watt is more power consumption, more heat and more noise (and with a server the noise can be a much bigger problem, depending on where it is). AMD's budget CPUs are great for these requirements.
 
* RAM: Any sane server OS will easily be okay with 512 MB. 1-2 GB doesn't hurt, but above 2 GB is overkill. (even 1 GB is overkill unless you're actually doing some heavy stuff)
 
* HDD: Anything big and cheap works fine. Even really shitty old drives can be repurposed and put in a suitable RAID, to compensate for failure, low speed or small capacity. This is probably the most critical spec of your server, besides power consumption - just stick every spare HDD you have in there.
 
* PSU: Your peak power consumption will probably be less than 100W, and you will never be at peak (maybe when installing OS). Unfortunately, it's hard to find decent PSUs (given that this machine is always powered, PSU is probably not a place to skimp) below 500W, so you will probably end up with those.
 
  
=== Operating system===
 
You should run [[Debian]], RHEL or CentOS if you want [[GNU/Linux]], or any [[BSD|BSD]] that you like.<br>
 
 
[[Ubuntu]] usually does retarded things with their packages and versions (lib*-ubuntu1.l2), and pulls unstable software from Debian Sid.
 
 
Rolling release distros ([[Arch]], Fedora) are not really good for a server, because it's supposed to stay working, and it shouldn't break/change it's behavior on updates.
 
 
[[Gentoo]] is usually too much trouble to be worth it, but it's OK.
 
 
You should also consider a NAS-centric operating system for a home server. FreeBSD-based FreeNAS or NAS4Free are common choices.  Both are [[free]] software and have simple GUIs to set up your services.
 
 
If you have a Raspberry Pi that you want to be put to use, ArkOS is a stable, Arch based distro for running a home server on a Raspberry Pi with a web-based GUI. [https://arkos.io ArkOS main website]
 
 
== VPS ==
 
A VPS is a virtual private server. When you rent a server from a company, they don't literally go and build a new machine just for you. They have huge server boxes running a VM software, and they just create a new virtual machine for you. That is your VPS.
 
 
=== Companies===
 
[https://trac.torproject.org/projects/tor/wiki/doc/GoodBadISPs A handy link to decide.]
 
 
=Setting up your services=
 
  
 
== Centralized storage==
 
== Centralized storage==
Line 84: Line 66:
 
You may want to consider a [[RAID]] array for long-term file storage.  RAID is not backup, but will protect your files in case of drive failure.  NAS4Free allows you to easily set up RAID arrays using UFS or ZFS.
 
You may want to consider a [[RAID]] array for long-term file storage.  RAID is not backup, but will protect your files in case of drive failure.  NAS4Free allows you to easily set up RAID arrays using UFS or ZFS.
  
== Cloud storage ==
+
== Web server ==
You can store files on your server, so that they are accessible from anywhere (but [[Passwords | password]] protected). These services are aimed at gaining access from the WAN (Wide Area Network, aka the internet). These services can be used on a LAN, but are cumbersome at times for most uses.
+
[[File:Tidle town.png|thumb|right|alt=A reminder why you should always self-host and if you don't, avoid inbred retards|A reminder why you should always self-host and if you don't, avoid inbred retards]]
 
 
=== FTP ===
 
 
 
FTP refers to a protocol specification that stands for File Transfer Protocol. This is a protocol that allows uploading, downloading, moving, and renaming files on a remote system. This simple protocol that is supported in a large number of places but offers a reasonable amount of security to the remote system. There are not automated aspects to this, as it is just a protocol, but there are plenty of tools that work on this standard to do work for you (notably FileZilla for beginners)
 
 
 
=== Seafile ===
 
  
A file hosting software system. Files are stored on a central server and can by synchronized with personal computers and mobile devices via the Seafile client. Files can also be accessed via the server's web interface. Seafile's functionality is similar to other popular services such as Dropbox and Google Drive, with the primary difference being that Seafile is free and open-source, enabling users to host their own Seafile servers without artificially imposed limits on storage space or client connections.
 
 
=== OwnCloud ===
 
 
OwnCloud is essentially your own Dropbox. You run the OwnCloud server program on your server to make the cloud available, and create accounts for users. The rest is analogous to Dropbox: Every machine that wants to access the cloud installs the OwnCloud client, puts in your server address, their OwnCloud username/password and choose a directory to sync.
 
 
As of early 2015, the files are stored on the server's hard drive without any extra encryption. This means that the admin of the server (ie. you) can see the files of all clients. If you only want to sync your own files, that's fine. But if, say, you want to let your friend use your server instead of Dropbox then this potentially allows you to eavesdrop on what your friend puts in his cloud. Granted, this is exactly the same as Dropbox Inc being able to see what you put on your Dropbox.
 
 
Usually after setting up OwnCloud you also set up a web server like apache which provides a web interface at something like http[s]://server.com/owncloud. This is analogous to the Dropbox web UI, and allows users to do things like share files with others, manage their cloud files without running a client/syncing, and so on. It also lets the server admin (probably you) manage the accounts and various settings.
 
 
OwnCloud also keeps file history data (changes/deletions) in a SQL database, so usually setup involves getting MySQL running on your server as well.
 
 
== Web server ==
 
 
A web server serves up a page. The nice things about serving it from a server, than, say, Wordpress or your Dropbox share, is that now you can run web apps and server side code for a dynamic page.
 
A web server serves up a page. The nice things about serving it from a server, than, say, Wordpress or your Dropbox share, is that now you can run web apps and server side code for a dynamic page.
  
Keep in mind that many server software installs (such as the email server below) involve setting up a webserver, so you may not even need to do anything.
+
Keep in mind that many server software installs (such as email with) involve setting up a webserver, so you may not even need to do anything.
 
 
===Apache vs. nginx ===
 
  
 
===HTTPS===
 
===HTTPS===
The extra CPU burden of SSL is minuscule. Your server should serve up everything on HTTPS only. Keep port 80 (plain HTTP) open but redirect everything to HTTPS. If port 80 is closed, typing the address of your server into the address bar of a browser will probably fail (because the browser assumes you meant HTTP, but you have to go to HTTPS).
+
The extra CPU burden of TLS is minuscule. Your server should serve up everything on HTTPS only. Keep port 80 (plain HTTP) open but redirect everything to HTTPS. If port 80 is closed, typing the address of your server into the address bar of a browser will probably fail (because the browser assumes you meant HTTP, but you have to go to HTTPS).
  
Issue a self-signed certificate. CAs are for jerks. Set the duration short (eg. a year) and don't forget to make a new one.
+
Issue a self-signed certificate. CAs are for jerks. Set the duration short (eg. a year) and don't forget to make a new one. If you've got a domain, get a Lets Encrypt-signed cert and set up a cron job to renew it. They're pretty sweet.
If you've got a domain, get a Lets Encrypt-signed cert and set up a cron job to renew it. They're pretty sweet.
 
  
== Setting Up Email the EASY way ==
+
[https://certbot.eff.org/ Certbot] makes https easy to implement with Let's Encrypt certificates
 
 
''See also:  [[Email]]''
 
 
 
Want to use your own email server to avoid the [[NSA]]?  Good call!  But setting up email servers can be pretty complicated.  Assuming you mostly don't know what the hell you are doing, and assuming you're already secured your system per above, have a peek at [http://www.iredmail.org/ iRedMail].  iRedMail is an automated email and web server setup package.  It works best if installed on a FRESH system - if you're already fumbled around with Apache and/or dovecot and/or postfix and failed, wipe your shit and start over with iRedMail.  It will install and configure Postfix, Dovecot, Apache, and MySQL.  It also installs and configures fail2ban and iptables.  It includes spam filtering and greylisting.  It just works.  Its pretty awesome. 
 
 
 
You will, however, still need to manually set up your DNS records (MX, SPF, and DKIM).  Refer to the [[Email]] article for more on this.
 
 
 
If you want to get fancy and replace MySQL with MariaDB, or replace Apache with, say, Nginx, you can do that after you set up iRedMail, but any breakage is up to you to fix.
 
  
 
==Remote access via SSH==
 
==Remote access via SSH==
Line 139: Line 91:
 
:* <code>systemctl enable sshd.service && systemctl start sshd</code>, if you use systemd.
 
:* <code>systemctl enable sshd.service && systemctl start sshd</code>, if you use systemd.
  
3. If that does not work, you need to install <code>sshd</code> with your package manager.
+
3. If that does not work, you need to install <code>sshd</code> or <code>openssh-server</code> with your package manager.
 
 
=== On [[Windows]] ===
 
  
* Download [http://cygwin.com/install.html Cygwin]
 
* Choose "Install From Internet"
 
* Choose a mirror to download from
 
* When you get to the "Select Packages" screen, search SSH
 
* Choose to install the "Bin" version of the "openssh" package
 
* Download all dependencies and wait for setup to complete
 
* Open Cygwin and wait for the initial setup to complete
 
* Type "ssh-host-config" without the quotes
 
* Say yes when you get asked about privilege separation
 
* Say yes when asked to create a local user named sshd
 
* Say yes when asked to install sshd as a service
 
* When you see "CYGWIN=", type "ntsec tty" without the quotes ([http://www.chinese-watercolor.com/LRP/printsrv/cygwin-sshd.html this guide]explains why)
 
* You should be done now setting up the SSH server. To start it, type "net start sshd" without the quotes.
 
 
If you see:
 
 
The CYGWIN sshd service is starting.
 
The CYGWIN sshd service was started successfully.
 
 
Then everything went correctly. If not, you should Google the error.
 
  
 
=== On [[OS X]] ===
 
=== On [[OS X]] ===
Line 184: Line 114:
 
====Compression====
 
====Compression====
 
[http://freecode.com/projects/ziproxy Ziproxy] (Opera style web compression, including images)
 
[http://freecode.com/projects/ziproxy Ziproxy] (Opera style web compression, including images)
 
== Domain and static/dynamic IP ==
 
The primary identifier of your server on the internet is the IP. Not only is it a huge pain to memorize and type the IP every time, but most likely your IP isn't static, and you get a new one from your ISP periodically.
 
 
It's a lot more convenient to have a domain like server.mydomain.com that you can use to find the server. Normally, to get a domain you need to pay, and all the good ones are already taken. But if you don't mind some extra stuff in your URL, you can use a dynamic DNS.
 
 
=== [https://freedns.afraid.org/ FreeDNS] ===
 
Make a free account, then go to subdomains and click [https://freedns.afraid.org/subdomain/edit.php add]. Type in what subdomain you want to use, and pick one of the free domains. For example, if you want your server to be called cocks, and you pick the domain chickenkiller.com, your server will be accessible at cocks.chickenkiller.com (although this one is probably taken already). Your IP should be automatically detected, but if not, open http://ident.me/ from your server to find out what it is.
 
 
Click save, and it should pop-up in your subdomains. After a few seconds, the URL should point to your server's IP. You can test with SSH, ping, or opening in a web browser depending on what you have working. However, if your IP should change, you will need to manually update it. For doing this easily, FreeDNS actually provides an [https://freedns.afraid.org/api/ API]. If you click the "ASCII" link there, you should get a table of all your domains. At the end of each line is an address. Every time you load this address in a browser, the IP will get updated.
 
 
The Python script below checks the current IP, compares to the FreeDNS's IP, and updates if they differ every time it's run. It also creates a log file in whatever directory it's in, letting you inspect past operation.
 
 
<nowiki># Domains that should point to the current IP
 
freedns_domains = set(["mysubdomain.mydomain.com"])
 
 
# FreeDNS API endpoint, accessible from https://freedns.afraid.org/api/ (use the ASCII one)
 
freedns_check_url = "https://freedns.afraid.org/api/?action=getdyndns&sha=API_KEY_GOES_HERE"
 
 
# It is assumed that this address responds with a page containing only the IP address
 
ip_check_url = "https://ident.me/"
 
 
 
import logging
 
logging.basicConfig(filename="log.txt",
 
                    level=logging.DEBUG,
 
                    format="%(asctime)s:  %(message)s",
 
                    datefmt="%m/%d/%Y %I:%M:%S")
 
logging.info("")
 
logging.info("Script executing...")
 
 
import urllib.request
 
current_ip = urllib.request.urlopen(ip_check_url).read().decode("utf-8")
 
logging.info("Current IP is " + current_ip)
 
 
logging.info("Getting FreeDNS records...")
 
import urllib.request
 
records_raw = urllib.request.urlopen(freedns_check_url).read().decode("utf-8").split("\n")
 
logging.info("Found " + str(len(records_raw)) + ".")
 
 
for raw_record in records_raw:
 
    domain, current_freedns_ip, freedns_update_url = raw_record.split("|")
 
 
    if domain not in freedns_domains:
 
        logging.info(domain + " is not on the watch list, skipping.")
 
    else:
 
        logging.info(domain + " (" + current_freedns_ip + ") is on the watch list.")
 
 
        if current_ip == current_freedns_ip:
 
            logging.info("IPs already match.")
 
        else:
 
            logging.info("IPs don't match, updating...")
 
            urllib.request.urlopen(freedns_update_url)
 
            logging.info("Done.")
 
 
logging.info("Script exited gracefully.") </nowiki>
 
 
Put this somewhere on your server, such as <code>~/scripts/freedns/freedns_update.py</code>. You can test by running it manually (<code>python3 ~/scripts/freedns/freedns_update.py</code>) and you can check the log (<code>nano ~/scripts/freedns/log.txt</code>). To automatically run the script, set up a cron job: <code>crontab -e</code> then add the following line:
 
 
0 9 * * * python3 ~/scripts/freedns/freedns_update.py
 
 
This will run the script at 9 AM every day. In principle, you could just make a cron job (<code>wget https://freedns.afraid.org/api/?action=getdyndns&sha=API_KEY_GOES_HERE</code>) for each domain, but that way you wouldn't get a log.
 
  
 
==Media automation==
 
==Media automation==
Line 258: Line 126:
 
You can use a daemon like [https://github.com/rembo10/headphones/ Headphones]
 
You can use a daemon like [https://github.com/rembo10/headphones/ Headphones]
  
[[Setting_up_a_Server/Security]]
+
=Security=
 +
[[Setting up a Server/Security]]
  
 
= External links =
 
= External links =

Latest revision as of 03:05, 20 March 2019

Need a fileserver that won't face the external internet? Check Home server.

Need to email? Set up a web server? Well, here's some advice. We're gonna try to write this page like you've never done this shit before. It does, however, assume you have at least some basic GNU/Linux knowledge. If you don't, you probably aren't ready for this. You weren't going to set up a server using Windows, were you? Jesus Christ, how horrifying.

A lot of this applies to both a physical machine as well as a VPS setup.

Common uses for a server

  • Install FTP software and run an FTP site.
  • Install a cloud service like Seafile to run your own Dropbox service, no privacy issues, full control, unlimited space (well, limited by how many drives you can cram in).
  • Always on seedbox. Start torrents with your phone through the web interface while out, they're done by the time you're back home.
  • Host a personal website.
  • Run your own mailserver just like Hillary!
    • Warning: Running a mailserver is a shitton of work. You will get hacked all the fucking time and it's very high-maintenance.
  • Run a dedicated game server.
  • Run various webapps, develop your own webapps.
  • SSH-tunnel to the server from work/school/etc to use it as a proxy, so that the admin of the network you're on can't see what sites you're going on.
  • Run a VPN for location spoofing or security when you're out and about.

Getting Started

Home or Remote?

Choosing an Operating System

Setting up your services

Most packages have clear tutorials on their repo/project site. Here are some handpicked guides as well as some written by other anons.

  • Mail
  • DNS
  • FTP
  • Cloud Storage
  • Web Server
  • ZNC
  • SSH
  • Everything in Docker
  • HAProxy (for multiple machines behind one ip).
  • VPN
  • Media Automation (Sickbeard, Flexget etc.)
  • Security
  • Media Streaming with icecast/ffmpeg/cvlc

Recommended software

  • Mail: see Setting up a Server/Mail
  • DNS: see Setting up a Server/DNS#Authoritative
  • FTP: vsftpd, glftpd (nonfree)
  • IRCd: inspircd, ngircd, kike
  • IRC Bouncer: znc
  • XDCC: iroffer-dinoex
  • XMPP: Prosody
  • Proxy: danted
  • Usenet Server: Leafnode, InterNetNews
  • Web Cache/Reverse Proxy: varnish, squid, nginx
  • Web: nginx, darkhttpd (use stunnel for ssl)
  • Config Management: cdist, ansible, salt
  • Control Panel: Webmin, Cockpit
  • Direct Connect Server (DCC): uhub, luadch
  • Monitoring:

Less common

  • BBS Server: Mystic BBS (BBSs are dead)
  • MUD: PennMUSH



Centralized storage

A server is perfect for this job. It is (supposedly) an always available resource on the local network. If using this in your house, you can expect reasonable speeds, even over WiFi that will let you do many daily tasks. One option is to set it up with NFS (Linux-centric, can be used on windows but it's shit) or samba, so you can watch your chinese cartoons on any device and keep your documents/whatever synchronized. This synchronization is a key benefit of the network storage.

You may want to consider a RAID array for long-term file storage. RAID is not backup, but will protect your files in case of drive failure. NAS4Free allows you to easily set up RAID arrays using UFS or ZFS.

Web server

A reminder why you should always self-host and if you don't, avoid inbred retards
A reminder why you should always self-host and if you don't, avoid inbred retards

A web server serves up a page. The nice things about serving it from a server, than, say, Wordpress or your Dropbox share, is that now you can run web apps and server side code for a dynamic page.

Keep in mind that many server software installs (such as email with) involve setting up a webserver, so you may not even need to do anything.

HTTPS

The extra CPU burden of TLS is minuscule. Your server should serve up everything on HTTPS only. Keep port 80 (plain HTTP) open but redirect everything to HTTPS. If port 80 is closed, typing the address of your server into the address bar of a browser will probably fail (because the browser assumes you meant HTTP, but you have to go to HTTPS).

Issue a self-signed certificate. CAs are for jerks. Set the duration short (eg. a year) and don't forget to make a new one. If you've got a domain, get a Lets Encrypt-signed cert and set up a cron job to renew it. They're pretty sweet.

Certbot makes https easy to implement with Let's Encrypt certificates

Remote access via SSH

GNU or BSD based systems

1. You usually enable the ssh server during the installation. Do this if possible, it is the simplest way.

2. If you did not setup sshd to auto start you can type:

  • /etc/rc.d sshd start, if you use System V init.
  • systemctl enable sshd.service && systemctl start sshd, if you use systemd.

3. If that does not work, you need to install sshd or openssh-server with your package manager.


On OS X

Simply go into sharing permissions and enable fucking everything.

Post-setup

Tunneling

Create a tunnel and use it as a proxy for environments that block certain DNS requests or pages and to encrypt your data

Wake on LAN

Turn on a PC on your LAN Arch Wiki guide

Web hosting

Host webpages, use nginx or apache debian nginx guide

Proxy

You can use a proxy guide

Compression

Ziproxy (Opera style web compression, including images)

Media automation

Torrenting

Use a daemon like Transmission or Deluge.

TV Series

You can use a daemon like Sickbeard

Movies

You can use a daemon like Couchpotato

Music

You can use a daemon like Headphones

Security

Setting up a Server/Security

External links

Linode Library - Good beginner tutorials.