How to Setup Sandstorm Personal Cloud Server in Linux

Sandstorm is an Open Source self-hostable web productivity suite implemented as a security-hardened web app package manager. It is a radically easier way to run personal instances of your web application at one place. It allows you to have your own personal server to install multiple application on it through an app store interface as easily as you would install apps on a phone. Sandstorm keeps a list so you can find everything you create and its unified access control system covers data from every app, and everything is private to you by default. Find any app you want on the App Market and start using it with a few clicks. Every app comes with automatic updates. More than all it protects you, each document, chat room, mail box, notebook, blog, or anything else you create is a “grain” in Sandstorm. It containerizes each one in its own secure sandbox from which it cannot talk to the world without express permission. All your grains are private until you share them. The result is that 95% of security vulnerabilities are automatically mitigated.

Prerequisites

To make Sandstorm run on CentOS 7, we will be required to have systems with following competencies.

  • Linux Kernel 3.10+
  • User namespaces disabled

According to its basic software requirements, you can easily install it on RHEL-7 or CentOS 7 as both have the kernel versions greater than 3.10. Like the same way if you have to install it on Arch Linux, you can do so because of its kernels compiles with ‘CONFIG_USER_NS=n’.

Other than software requirements, you can use 1GB+ of RAM but 2GB+ is recommended. Here in this article we will be using a CentOS 7.2 VM with 2GB RAM , 2 CPUs and 20 GB disk space.

How to update your system

Once you have access to the VM, create a non-root user with sudo privileges to perform all system level tasks. In CentOS 7 you create a new user with sudo rights using below commands.

$ ssh root@server_ip

# adduser new_user

Set your password for the new user, and then Use the ‘usermod’ command to add the user to the ‘wheel’ group.

# usermod -aG wheel new_user

Now using the ‘su’ command, switch to the new user account and run the command with sudo to update your system.

# su – new_user

# sudo yum update -y

After system update with latest updates and security patches, move to the next step to download and install the Sandstorm on CentOS 7.

How to install Sandstorm

This comes with its own installer that provides its automatic installation setup. To install on your own Linux machine, you just need to run below ‘curl’ command.

$ curl https://install.sandstorm.io | bash

Then You can have two options, and you need to choose the appropriate one, either you like to go for 1 or 2 .

1. A typical install, to use Sandstorm (press enter to accept this default)
2. A development server, for working on Sandstorm itself or localhost-based app development

Let’s choose the option ‘1’ and press Enter key to go for its default typical installation.

This complete installation setup with go through the following process.:

* Install Sandstorm in /opt/sandstorm
* Automatically keep Sandstorm up-to-date
* Configure auto-renewing HTTPS if you use a subdomain of sandcats.io
* Create a service user (sandstorm) that owns Sandstorm’s files
* Configure Sandstorm to start on system boot (with systemd)
* Listen for inbound email on port 25.

To set up Sandstorm, we have to provide the sudo privileges, type ‘yes’ to allow sudo access to continue after its password.

Note that Sandstorm’s storage will only be accessible to the group ‘sandstorm’. As a Sandstorm user, you are invited to use a free Internet hostname as a subdomain of sandcats.io, a service operated by the Sandstorm development team. You can choose your desired Sandcats subdomain (alphanumeric, max 20 characters). Type the word ‘none; to skip this step, or ‘help’ for help.

What *.sandcats.io subdomain would you like? [] linox

Next you need to mention your email on file so it help you recover your domain if you lose access.

Enter your email address: [] kashifs@linoxide.com

This register your domain, and you will be provided with a URL that users will enter in browser.

[http://linox.sandicats.io:6080]

Next Sandstorm requires you to set up a wildcard DNS entry pointing at the server. This allows Sandstorm to allocate new hosts on-the-fly for sandboxing purposes. Please enter a DNS hostname containing a ‘*’ which maps to your server. For example, if you have mapped *.foo.example.com to your server, you could enter “*.foo.example.com”. You can also specify that hosts should have a special prefix, like “ss-*.foo.example.com”. Note that if your server’s main page is served over SSL, the wildcard address must support SSL as well, which implies that you must have a wildcard certificate. For local-machine servers, we have mapped *.local.sandstorm.io to 127.0.0.1 for your convenience, so you can use “*.local.sandstorm.io” here. If you are serving off a non-standard port, you must include it here as well.

Wildcard host: [*.linox.sandicats.io:6080] *.linox.sandicats.io.com

Server installation is complete now, Visit the link mentioned in the end of the setup to start using it.

http://ksh-cen-7.domain.com:6080/setup/token/36d4f17a3804ba7e19cc159a844f3e45e7a726c5

installation

As mentioned the URL expires in 15 minutes. You can generate a new setup URL by running below command.

$ sudo sandstorm admin-token

session token

How to configure Sandstorm Web setup

Once you open the URL, you will see a welcome page to begin the admin settings and to configure your login system.

welcome sandstorm

1) Identity providers

To use Sandstorm, you need to create a user account. Every user account on Sandstorm is backed by an identity provider. You’ll use this identity provider to authenticate as the first administrator of this Sandstorm install.

Configure the identity provider or providers you wish to enable by a click on the ‘configure’ button.

identity provider

Let’s see if you want to enable Github on your Sandstorm, click on the configure button, a new window will be opened where you need to provide github login configurations. Once you got your Client ID and Client secret from your github account, click on the ‘Enable’ button to proceed.

github configuration

2) Organization settings

Sandstorm allows you to define an organization. You can automatically apply some settings to all members of your organization. Users within the organization will automatically be able to log in, install apps, and create grains.

Organization settings

3) Email delivery

Sandstorm needs a way to send email. You can skip this step (unless you’re using email login), but email-related features will be unavailable until you configure email in the future. Mention your SMTP host with Port and credentials.

email delivery

4) Pre-installed apps

Here Sandstorm installs the following Productivity Suite apps that are useful for most users shown below. You will be able to configure all pre-installed apps in the Admin Settings panel after setup.

pre install app

5) Create Admin account

Log with your google or Github account that you created in previous step to create your admin account.

admin account

That’s it, now add more users, edit other settings or start user your awesome personal cloud platform.

start using

Conclusion

In the end of this article, you are now able to install, configure and use your own personal cloud platform on CentOS 7. It aims to tackle the authentication and security problems that software-As-A-Service poses for many companies through the use of fine-grained containerization. Using Sandstorm now it’s much easier than setting up yourself because you just to point and click, your click install and you have the app running. It takes like 5 seconds to spin up a container that help’s you build your own applications within seconds.

Advertisements

Pet – A Command Line Snippet Manager for Linux

If you manage a lot of Linux systems via command line, you must be aware of the hectic task to remember all the commands you use on daily basis. You often need to go into your system’s history and find the required commands from there. Pet is a command line based tool which makes a system administrator’s life easy. It is an easy to use snippet manager, which saves your command snippets and you can easily view and use them when needed. Command line junkies find this tool extremely important, it is completely a freeware script, the setup process is pretty simple and it runs on almost all flavors of Linux operating system as well as Mac OS. It is built using the well known Go programming language and requires some python modules like Peco and brew to work properly. It is written by Teppei Fukuda and its source code is available on Github. In this tutorial, we will discuss its prominent features, installation process, and some usage example. Stay with us, it’s going to be an interesting session.

Important Features of Pet

Here are some of the noteworthy features of this utility, it can:

  • register your new command snippets pretty quickly.
  • search through the local repository of command snippets.
  • easily edit and run command snippets.
  • use Gist (GitHub-based online code repository) to sync your snippets.

Feeling excited? Let’s get started on its installation and configuration process now.

Installing and Using Pet

We will be demonstrating the installation and usage process of Pet on Ubuntu 16.10 system, the same set of instructions should work for any older version of Ubuntu and Debian based systems. As mentioned in the introductory paragraph, it requires Peco and brew to work properly. Run following two commands respectively to install Brew and Peco on your Linux system.

sudo apt-get install linuxbrew-wrapper
sudo brew install peco

Once the pre-requisites have been properly installed, run following command to download the source file for Pet.

sudo wget https://github.com/knqyf263/pet/releases/download/v0.0.2/pet_0.0.2_linux_amd64.zip

Depending on your network speed, it should take a couple of minutes to complete the download process. Run the following mentioned command to unzip the downloaded file.

unzip pet_0.0.2_linux_amd64.zip

Once the unzip process is complete, you should be able to see an executable script, named “pet” here. Run the following command to copy this script file to proper location so it may be available for use for all users.

sudo cp pet /usr/local/bin

There you go, Pet has been configured successfully now. Run “pet” command on the terminal to verify that it is working properly. Expected output should be as depicted in the following screenshot.

pet

Let’s demonstrate some examples, in order to add the new snippets, use the following command:

pet new

It will prompt you the two details, “Command” and “Description”. I have following command I often use on my Linux web servers to identify if my server is under DDOS attack or not.

netstat -nap | grep \:80\  | awk ‘{print $5}’ | tr “:” ” ” | awk ‘{print $1}’ | sort | uniq -c | sort -n

Below screenshot will clarify how I added this snippet to Pet.

pet add

Similarly, you can use the following command to list the currently added snippets.

pet list

You can search through added snippets using the following command:

pet search

You can easily register the previously executed commands to the pet database by edit your .zshrc file. Use vi or vim editor to open the .zshrc file and append the following lines there:

function prev() {
PREV=$(fc -lrn | head -n 1)
sh -c “pet new `printf %q “$PREV”`”
}

Now you don’t need to add each command manually to Pet, your system will automatically add all the commands you use to the snippet manager. Your daily work life is lot easy now 🙂

Sync Snippets to Gist

As mentioned in the feature list, you can sync your snippets with online Gist repository, for this purpose you must have a Github account, once you have GitHub account, you can get a GIST Token easily. Use the following command to sync your snippets to Gist, it will prompt for Token ID and you should be good to go.

 pet sync -u

Here is the sample output of this command:

 pet sync -u
Gist ID: 1P93IUdf4e06d117097en976BHY
Upload success

Similarly, you can download your snippets from Gist to your system using the following command. It is pretty useful when you want to migrate your snippet data to any other system.

pet sync

Hope you enjoyed this article, we have discussed all possible usages of Pet utility here. It is an extremely lightweight, easy to use, and stable utility which is a must-have application for any command line Linux lover. It is under continuous development and we hope to see many improvements to this application in near future. If you have any questions or feedback, fee free to let us know in the comments section of this article.

Microservices at Netflix scale

2017-03-11 18_37_43-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player2017-03-11 18_38_19-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player2017-03-11 18_40_13-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player2017-03-11 18_40_28-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player2017-03-11 18_40_45-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player

Netflix took 7 years to completely transform to microservices. The traditional approach that was followed was that developers contributed to their individual jars/wars which would go through the regular sprint iteration.

2017-03-11 18_42_47-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player.png

As can be seen above that there are issues with the velocity for delivery and reliability.

Any one change in one service it would reflect into the other services which was very difficult to handle. This caused too many bugs and single point database. Few years back the production database of netflix got corrupted and the users/customers saw the following message.

2017-03-11 18_47_48-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player.png

2017-03-11 18_49_46-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player2017-03-11 18_50_22-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player

2017-03-11 18_52_13-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player2017-03-11 18_52_53-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player

2017-03-11 18_54_05-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player2017-03-11 18_54_44-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player

2017-03-11 18_56_36-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player.png

2017-03-11 18_57_58-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player.png

2017-03-11 19_03_16-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player2017-03-11 18_58_26-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player2017-03-11 18_59_06-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player2017-03-11 18_59_39-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player2017-03-11 19_00_14-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player2017-03-11 19_00_27-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player2017-03-11 19_00_47-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player2017-03-11 19_01_09-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player2017-03-11 19_01_20-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player2017-03-11 19_01_40-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player2017-03-11 19_02_29-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player2017-03-11 19_02_57-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player

2017-03-11 19_07_25-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player2017-03-11 19_07_47-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player

2017-03-11 19_13_46-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player2017-03-11 19_10_19-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player2017-03-11 19_11_12-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player2017-03-11 19_11_25-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player2017-03-11 19_13_03-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player

At netflix they want their services to isolate single point failures so here comes Hystrix. Hystrix is a latency and fault tolerance library designed to isolate points of access to remote systems, services and 3rd party libraries, stop cascading failure and enable resilience in complex distributed systems where failure is inevitable.

They test their system with fault injection test framework (FIT).

2017-03-11 19_19_45-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player2017-03-11 19_20_10-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player2017-03-11 19_21_50-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player2017-03-11 19_21_43-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player2017-03-11 19_22_05-GOTO2016•Micros-Download-From-YTPak.com (1).mp4 - VLC media player