GIT commants I've found useful    Posted:


Check files changed between branches

git diff --name-status master..devel

Check changes on file form different branch/commit

git diff commit_hash -- filename

Same as above between 2 branches/commits

git diff commit_hash master -- filename

Check full file history

git log -p -- filename

Check who broke production server :

git blame filename

Merge as one commit (need to commit afterwards) its not default like in normal merge:

git merge --squash branch

List of commits in git local storage

git reflog

Take(checkout) file from different branch/commit

git checkout develop -- filename
git checkout commit_hash -- filename

Reset current branch to remote:

git reset --hard origin/current_branch
git reset --hard origin/master

Save and depracate changes which were not commited

git stash
git stash save -a

Restore stash (by picking selected)

git stash list
git stash pop {0}

Comments

fabric - auto deployment script    Posted:


Recently I wrote fabric deployment scrip maybe someone will find it usefull.

It enables possibility to run "group execute" task with

fab live_servers pull restart

or single host

fab live1 pull

All we need to do is to define group or sinle host as function afterwards I used end update decorator.

I know there could be also something like duplication of tasks with separate servers fab live1 pull live2 pull but I believe that fabric was written for distributed systems which has different paths of apps and users etc.

also roledefs with extra dict keys didn't work for me)? I want to keep this simple single/multiple host deployment commands like : fab live_servers pull, fab test pull

from fabric.api import run, env, local, get, cd
from fabric.tasks import execute
import inspect
import sys
import os
import re
from StringIO import StringIO

# fabfile author: Grzegorz Stencel
# usage:
# run: fab help for examples
# fab staging svnxapp:app=holdings_and_quotes,layout.py,permissions.py restart
# fab test svnxlib

SERVER_BRANCHES = {
    'live': 'master',
    'sit': 'sit',
    'uat': 'uat',
    'live2':'master',
    'live3':'master'

}
# MAIN CONF
SERVERS = {
    'local': {
        'envname': 'local',
        'user': 'greg',
        'host': 'localhost',
        'host_string': 'localhost',
        'path': os.environ.get('SITE_ROOT', '/opt/myapp/test'),
        'www_root': 'http://localhost:8081/',
        'retries_before_killing': 3,
        'retry_sleep': 2
    },
    'test': {
        'envname': 'test',
        'user': 'root',
        'host': 'myapp-test.stencel.com',
        'host_string': 'myapp-test.stencel.com',
        'path': '/var/www/myapp/test/',
        'www_root': 'http://myapp-test.stencel.com/',
        'retries_before_killing': 3,
        'retry_sleep': 2
    },
    'uat': {
        'envname': 'uat',
        'user': 'myapp',
        'host': 'uat.myapp2.stencel.com',
        'host_string': 'uat.myapp2.stencel.com',
        'key_filename': 'deploy/keys/id_rsa',
        'path': '/opt/myapp/uat/',
        'www_root': 'http://uat.myapp2.stencel.com/',
        'retries_before_killing': 3,
        'retry_sleep': 2
    },
    'sit': {
        'envname': 'sit',
        'user': 'myapp',
        'host': 'sit.myapp2.stencel.com',
        'host_string': 'sit.myapp2.stencel.com',
        'key_filename': 'deploy/keys/id_rsa',
        'path': '/opt/myapp/sit/',
        'www_root': 'http://sit.myapp2.stencel.com/',
        'retries_before_killing': 3,
        'retry_sleep': 2
    },
    'live': {
        'envname': 'live',
        'user': 'myapp',
        'host': '10.10.10.10',
        'host_string': 'myapp2.stencel.com',
        'path': '/opt/myapp/live/',
        'www_root': 'http://myapp2.stencel.com/',
        'retries_before_killing': 3,
        'retry_sleep': 2
    },
    'live2': {
        'envname': 'live2',
        'user': 'root',
        'host': '10.10.10.11',
        'host_string': 'live2.stencel.com',
        'path': '/var/www/myapp/live/',
        'www_root': 'http://myapp2.stencel.com/',
        'retries_before_killing': 3,
        'retry_sleep': 2
    },
    'live3': {
        'envname': 'live3',
        'user': 'root',
        'host': '10.10.10.12',
        'host_string': 'live3.stencel.com',
        'path': '/var/www/myapp/live/',
        'www_root': 'http://myapp2.stencel.com/',
        'retries_before_killing': 3,
        'retry_sleep': 2
    },

}

LIVE_HOSTS = ['live', 'live2', 'live3']


def list_hosts():
    """
    Lists available myapp hosts
    """
    print " Single hosts(if you want to pull from svn only to one of them):"
    print '   %s' % '\n   '.join([a for a in SERVERS])
    print " Multiple hosts"
    print '   live (which contains %s)' % ','.join([a for a in LIVE_HOSTS])


def test():
    """
    single host definition , "fab test restart" wil restart this one host

    """
    env.update(dict(SERVERS['test']))


def localhost():
    """
    single host definition , "fab test restart" wil restart this one host

    """
    env.update(dict(SERVERS['local']))


def uat():
    """
    single host definition , "fab uat restart" wil restart this single host

    """
    env.update(dict(SERVERS['uat']))


def sit():
    """
    single host

    """
    env.update(dict(SERVERS['sit']))


#  SERVERS GRcompanyS DEFINITION
def live():
    """
    multiple grcompany of hosts - running: "fab live restart" will restart all live servers

    """
    env['hosts'] = [SERVERS[a]['host'] for a in LIVE_HOSTS]

    # env.update(dict(SERVERS['staging']))


def env_update(func):
    """
    Decorator - needs to be added to each task in fabricfile - for multiple host task execution
    """

    def func_wrapper(*args, **kwargs):
        if not len(env.hosts):
            return func(*args, **kwargs)
        else:
            env.update(dict(SERVERS[filter(lambda x: SERVERS[x]['host'] == env.host, MyApp_SERVERS)[0]]))
            func(*args, **kwargs)

    return func_wrapper


@env_update
def bundle_media():
    """
    bundles media like css and js to one file.
    example:
        fab test bundle_media
    """
    # export DJANGO_SETTINGS_MODULE=settings
    #run("cd {0} && source settings/{1}-config.sh && python scripts/bundle_media.py".format(env.path,env.envname))
   run("source /usr/share/virtualenvwrapper/virtualenvwrapper.sh && workon {0} && python scripts/bundle_media.py".format("%s-myapp" % env.envname if env.envname<> 'live' else 'MyApp-test')) #change live venv to be live-MyApp

def _valid_branch(env):
    branch = run("cd {0} && git rev-parse --abbrev-ref HEAD".format(env.path))
    return branch == SERVER_BRANCHES[env.envname] and not env.envname=='local'


@env_update
def pull(*args, **kwargs):
    if _valid_branch(env):
        with cd(env.path):
            run("git fetch origin")
            run("git reset --hard origin/%s" % branch)
    else:
        print "Error : Server is checked out to wrong branch!!!"


            #run('git fetch --quiet')
            #run('git fetch --tags --quiet')

@env_update
def reload():
    """
    Reload specified servers - kills unused gunicorn workers but waits workers with old code to finish processing.

    """
    bundle_media()

    #if env.envname in ('uat', 'staging', 'live'):
    f = StringIO()
    get("/opt/myapp/%s/pid" % env.envname,f)
    pid = re.search(r'\d+',f.getvalue()).group()
    run("ps aux | grep gunicorn | grep %s | grep master | grep -v grep | awk '{print $2}'" % env.envname)
    run("kill -HUP %s" % pid)


@env_update
def restart():
    """
    Hard restarts specified servers

    """
    bundle_media()
    run("ps aux | grep gunicorn | grep %s | grep master | grep -v grep | awk '{print $2}'" % env.envname)
    run("supervisorctl stop myapp-%s && supervisorctl start MyApp-%s" % (env.envname,env.envname))
    run("ps aux | grep gunicorn | grep %s | grep master | grep -v grep | awk '{print $2}'" % env.envname)


def help():
    fabric_functions = ['run', 'execute', 'local', 'func_wrapper']
    functions = set([obj.__name__ if obj.__name__ not in fabric_functions else '' for name, obj in
                     inspect.getmembers(sys.modules[__name__]) if inspect.isfunction(obj)])
    functions.remove('')
    print "usage: \n  fab [host/grcompany of hosts] [commands] (optional command with arguments command:kwarg=val,arg1,arg2,arg3)"
    print "\navailable servers:"
    list_hosts()
    print "\ncommands:\n  %s" % ', '.join([a for a in functions])
    print "\nexamples:\n  staging svnxapp:app=holdings_and_quotes,layout.py,permissions.py restart"
    print "  fab test restart"
    print "  fab staging svnxapp:app=holdings_and_quotes,lib/quote.py,layout.py,models.py"
    print "  fab staging svnxapp:app=holdings_and_quotes,lib/quote.py restart"
    print "  fab test build"
    print "  fab test bundle_media restart"
    print " For svnx whole app (comma in the end):"
    print "  fab test svnxapp:app=medrep,"
    print " For global lib:"
    print "  fab test svnxlib"
    print " For whole global media:"
    print "  fab test svnxmedia:"
    print " For global media file:"
    print "  fab test svnxmedia:javascript"
    print "  fab test svnxmedia:javascript/company/checklist.js"
    print "\nIf .js file in args like : fab staging svnxapp:app=holdings_and_quotes,media/js/quote.js,layout.py,models.py"
    print "It will bundle media itself"
    print "Restart test staging without params:\n  fab restart"
    for f in functions:
        print f
        print globals()[f].__doc__
        print "\n"



@env_update
def accessguni():
    run("tail /var/log/myapp/access-%s.log" % env.envname.upper() )

@env_update
def accessgunilive():
    run("tail -f /var/log/myapp/access-%s.log" % env.envname.upper() )

@env_update
def errorguni():
    run("tail /var/log/myapp/error-%s.log" % env.envname.upper() )

@env_update
def errorgunilive():
    run("tail -f /var/log/myapp/error.log" % env.envname.upper() )

def hostname():
    run('uname -a')

@env_update
def uptime():
    run('uptime')

Comments

COMODO Positive-SSL sec_error_unknown_issuer on firefox    Posted:


Tempted by post on official google admins blog http://googlewebmastercentral.blogspot.co.uk/2014/08/https-as-ranking-signal.html it's about higher ranking when having https. So I have recently bought cheap ssl certificate from known issuer COMODO and upgraded my server with it but it caused several problems when I redirected from 80 to 443 some 3rd party apps (like userena for django) created loops but I found solution quickly it was to put USERENA_USE_HTTPS=True in settings.py but it still sends more forgot password mails than it should but it's different story.

Comming back to point if you are experiencing error "sec_error_unknown_issuer" in firefox after implementing COMODO Positive-SSL certificate this solution may help you. Don't know exactly but I read that it has something to do with chain certificates they should be bundled inside you website certificate. And remember you have to do it in right order because first one should be your servers certificate cause it's signed with key) and nginx or whatever you use may have problems after reloading service.

My command for bundling was:

cat exerceo_pl.crt COMODORSADomainValidationSecureServerCA.crt COMODORSAAddTrustCA.crt AddTrustExternalCARoot.crt > exerceo_pl.bundled.crt after that I had just to point my nginx.conf to new bundled certificate. And after all reaload of nginx fixed my unknow issuer firefox problem.

Comments

xrandr tips    Posted:


adding resolution 1)generate mode cvt 2560 1440 60 ➜ blog git:(website) ✗ cvt 2560 1440 60 # 2560x1440 59.96 Hz (CVT 3.69M9) hsync: 89.52 kHz; pclk: 312.25 MHz Modeline "2560x1440_60.00" 312.25 2560 2752 3024 3488 1440 1443 1448 1493 -hsync +vsync

2)add it to xrandr sudo xrandr --newmode "2560x1440_60.00" 312.25 2560 2752 3024 3488 1440 1443 1448 1493 -hsync +vsync setting resolution

3)sudo xrandr --addmode VGA-0 "1680x1050_60.00"

xrandr -q xrandr --verbose

xrandr --output HDMI-0 --mode 2560x1440

irst generate a "modeline" by using cvt Syntax is: cvt width height refreshrate

cvt 1680 1050 60 this gives you:

# 1680x1050 59.95 Hz (CVT 1.76MA) hsync: 65.29 kHz; pclk: 146.25 MHz Modeline "1680x1050_60.00" 146.25 1680 1784 1960 2240 1050 1053 1059 1089 -hsync +vsync Now tell this to xrandr:

sudo xrandr --newmode "1680x1050_60.00" 146.25 1680 1784 1960 2240 1050 1053 1059 1089 -

First clone the two screens, (the smaller screen will display the top left portion of the virtual screen)

xrandr --output VGA --auto --right-of LVDS

System Message: WARNING/2 (<string>, line 34)

Block quote ends without a blank line; unexpected unindent.

xrandr --output LVDS --mode 1280x800

xrandr --output LVDS --mode 1280x800 --rate 75

xrandr --output LVDS --auto xrandr --output LVDS --off --output HDMI-0 --auto

xrandr --output VGA1 --mode 1024x768 --rate 60

#Laptop right extra Monitor Left #xrandr --output VGA1 --left-of LVDS1

#Laptop left extra Monitor right xrandr --output LVDS1 --left-of VGA1

#This is to set your primary monitor.

#This sets your laptop monitor as your primary monitor. xrandr --output LVDS1 --primary

#This sets your VGA monitor as your primary monitor. #xrandr --output VGA1 --primary

xrandr --output VGA1 --mode 1024x768 --rate 60 xrandr --pos <x>x<y> $ xrandr --left-of <output> $ xrandr --right-of <output> $ xrandr --above <output> $ xrandr --below <output>

Option '-pos' is more flexible which can place output to anywhere, for example:

$ xrandr --output VGA1 --pos 200x200 $ xrandr --output LVDS1 --pos 400x500

xrandr -o right

Comments

Django Display choices defined in models in template    Posted:


example: Having model:

GENDER= (
    (1 , 'Male'),
    (0 , 'Female'),
)

class Profile(UserenaBaseProfile):
    name = models.CharField(max_length=100,null=True,blank=True)
    surname = models.CharField(max_length=100,null=True,blank=True)
    sex = models.NullBooleanField(blank=True,null=True,choices=GENDER)
    born = models.DateTimeField(blank=True,null=True)

    def __unicode__(self):
        return str(self.name)

we can display Male or Female choice value in template by:

{{profile.get_sex_display}}

Comments

docker on 32bit system (windows 7 on core2duo)    Posted:


Docker is very usefull tool for fast shipping application with its dependicies. It's totally different from Vagrant which I used before . Using them both may be very powerfull tool. So I started to play with docker but I had problem - my everyday environment is Debian but at my work I got 32bit core 2 duo workstation with windows7 installed (and no possibility to work on my laptop - security). Fortunatelly this processor has VT-x support so I could run 64bits virtual machines. I wanted to install boot to docker but unfortunatelly installation and boot2docker.exe are for 64 bits only so I just used their iso image because it was very lightweigh. First of all I had to convert this "live cd" image to vmdk I used this one

http://static.dockerfiles.io/boot2docker-v1.2.0-virtualbox-guest-additions-v4.3.14.iso qemu-img convert -f raw -O vmdk Nowyfolder/boot2docker.iso boot2docker-disk.vmdk

Now we have ready boot2docker disk out of live image. Start virtualbox, create new virtualmachine, select linux 64bits, and when there will be step with disk choose Existing disk option and select newly created boot2docker-disk.vmdk

after all I could start to create new virtualbox machine and select vmdk disk to existing one boot2docker-disk.vmdk we have ready and working virtual boot2docker machine .

Comments

docker    Posted:


First download download pre build image :

sudo docker pull ubuntu

run pseudo tty:

sudo docker run -t -i ubuntu /bin/bash

get image of running docker:

sudo docker ps

output: ➜ ~ sudo docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES c4fbc80d8926 ubuntu:14.04 /bin/bash 3 minutes ago Up 3 minutes berserk_nobel ➜ ~ save container as your image: ============================= sudo docker commit c4fbc80d8926 myimage_name

Running our image:

sudo docker run -t -i myimage_name /bin/bash

Deploying your app and dependicies:

watch out to save container before you quit bash inside docker instance find container id and save

Diff image container:

sudo docker diff

Start docker container with name:

sudo docker run -i -t --name="IWAPPS2" iwapps /bin/bash

Commit with name :

sudo docker commit IWAPPS2 local -a "greg" -m "First image commit"

Actually, images are stored within /var/lib/docker/aufs/diff

OUP sudo docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 95585eb7ea3c cc68531a3552 /bin/bash About a minute ago Up About a minute mad_ptolemy ➜ OUP sudo docker commit 95585eb7ea3c iwps2 -a "greg" -m "First image commit" c86eeb858bedf779105f759ca62d10ea59e01884e985e447d9ddf7f65ee32e3c ➜ OUP sudo docker images REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE iwps2 latest c86eeb858bed 5 seconds ago 617.5 MB local latest 539a99ee88ee 22 minutes ago 348.5 MB <none> <none> d8f374aecae4 2 hours ago 348.1 MB iwapps latest 85e0ae955062 4 hours ago 348.1 MB ubuntu utopic b8d67033ed07 23 hours ago 286.6 MB ubuntu 14.10 b8d67033ed07 23 hours ago 286.6 MB ubuntu trusty 1357f421be38 23 hours ago 192.7 MB ubuntu latest 1357f421be38 23 hours ago 192.7 MB ubuntu 14.04 1357f421be38 23 hours ago 192.7 MB ubuntu 14.04.1 1357f421be38 23 hours ago 192.7 MB ubuntu 12.04 2bed76595591 23 hours ago 120.2 MB ubuntu 12.04.5 2bed76595591 23 hours ago 120.2 MB ubuntu precise 2bed76595591 23 hours ago 120.2 MB <none> <none> 96864a7d2df3 3 weeks ago 205.1 MB ubuntu 12.10 c5881f11ded9 3 months ago 172.2 MB ubuntu quantal c5881f11ded9 3 months ago 172.2 MB ubuntu raring 463ff6be4238 3 months ago 169.4 MB ubuntu 13.04 463ff6be4238 3 months ago 169.4 MB ubuntu saucy 195eb90b5349 3 months ago 184.7 MB ubuntu 13.10 195eb90b5349 3 months ago 184.7 MB ubun

pt The images are stored in /var/lib/docker/graph/<id>/layer.

Note that images are just diffs from the parent image. The parent ID is stored with the image's metadata /var/lib/docker/graph/<id>/json.

When you docker run an image. AUFS will 'merge' all layers into one usable file system.

docker inspect docker images --no-trunc=true

Running an interactive shell sudo docker run -i -t ubuntu /bin/bash

sudo baobab /var/lib/docker

docker pull ubuntu docker run -i -t ubuntu:14.04 /bin/bash sudo docker info sudo docker version sudo docker search ubuntu sudo docker pull ubuntu sudo docker images

Comments