Python processes: some another way

signExit = False

pCount = 0

while not signExit:

for p in processes:

if p.successful:

print p.get()

pCount += 1

if pCount == numTasks:

break

print pCount

broker for celeryd

While start celeryd we can set BROKER_URL (address for give tasks and get it revert):

-b 'librabbitmq://guest:guest@ourProject.ru:5672//'

python: delete files by regex

import os, re

def purge(dir, pattern):
    for f in os.listdir(dir):
     if re.search(pattern, f):
      os.remove(os.path.join(dir, f))

ubuntu: update environment variables

source /etc/environment

neo4j using example: routing

a360.ru - our project that use routing based on neo4j.

PyQt: file save dialog (in windows)

make class:

class OpenFile(QtGui.QMainWindow):

   def __init__(self, parent=None):

       QtGui.QMainWindow.__init__(self, parent)

       self.setGeometry(300, 300, 350, 300)

       self.setWindowTitle('SaveFile')

       self.textEdit = QtGui.QTextEdit()

       self.setCentralWidget(self.textEdit)

       self.statusBar()

       self.setFocus()

       exit = QtGui.QAction(QtGui.QIcon('open.png'), 'Open', self)

       exit.setShortcut('Ctrl+O')

       exit.setStatusTip('Open new File')

       self.connect(exit, QtCore.SIGNAL('triggered()'), self.showDialog)

       menubar = self.menuBar()

       file = menubar.addMenu('&File')

       file.addAction(exit)

   def showDialog(self,text):

       filename = QtGui.QFileDialog.getSaveFileName(self, 'Save file', '')

       f=open(filename.toUtf8().data().decode('utf-8').encode('cp1251'),'w')

       f.write(text.toUtf8().data().decode('utf-8').encode('cp1251'))

     

in MainWindow class writing:

    def __init__(self, win_parent = None):

        ...

        #save stats in file

        self.connect(self.saveButton, QtCore.SIGNAL("clicked()"), self.saveFile)

    def saveFile(self):

        cd = OpenFile()

        cd.showDialog(self.textStatistic.toPlainText())

Django: django-admin - user was not authenticated

Problem: //localhost:8000/admin => error "user was not authenticated".
Solve: in postgreSQL - add this user as superuser; restart DB.

Django: default values in views

in urls:

...

    (r'^/test/$',test_func),

    (r'^/test/(\d+)/$',test_func),

...

in views:

def test_func(request, ddd='0'):

...

PyQt: pause in child thread

while (main_window.paused):

    time.sleep(1)

main_window.paused changing by current button.

stdin stdout in Python

import sys
import time
import copy
import csv
import re

while True:

#get string from stdin

    s=sys.stdin.readline()
    sss=[str(word).strip() for word in s.splitlines(False)]

#if string="Q", than exit

    if sss[0]=='Q': break
    try:
        for ss in sss:

#with regexp parse string

            pattern=re.compile('^(\d{6});(\d{6})$')
            u=pattern.search(ss).groups()
            try:

#data to our function
                seq1,seq2=getSeq(u[0],u[1])
            except:
                stseq1=list()
                trseq1=list()

#send results to stdout
            print(';'.join(str(j) for j in seq1) ,sep=';', end='\n', file=sys.stdout)
            print(';'.join(str(j) for j in seq2) ,sep=';', end='\n', file=sys.stdout)

XML in Python

#import

from xml.dom import minidom

#parse file

xmldoc=minidom.parse('input.xml')

#get element by tag name

#xmldoc.getElementsByTagName('ID')

#element as list

#xmldoc.getElementsByTagName('ID')[0]

#get child node

#xmldoc.getElementsByTagName('ID')[0].childNodes[0]

#some example

#loop by all child nodes of element with name 'Fragments'

for i in reversed(xmldoc.getElementsByTagName('Fragments')[0].childNodes):

#only elements with name 'Fragment'

    if i.nodeName=='Fragment':

#"number of fragment" - from attribute 'Num'

        fragmentNum=i.getAttribute('Num')

#list in child node

        for j in i.getElementsByTagName('RPs')[0].childNodes:

            if j.nodeName=='RP':

                for u in j.childNodes:

                    if u.nodeName=='Times':

#choose attribute of the node

                        i1=j.getAttribute('Pgsz')

knockout: standard example changing

Example for data auto change:

<!-- This is a *view* - HTML markup that defines the appearance of your UI -->

<p>First name: <input data-bind="value: firstName" /></p>
<p>Last name: <input data-bind="value: lastName" /></p>
<p>Full name: <strong id="txt">text</strong></p>
<button id="go">Update</button>

// This is a simple *viewmodel* - JavaScript that defines the data and behavior of your UI
function AppViewModel() {
    this.firstName = ko.observable("Bert");
    this.lastName = ko.observable("Bertington");
}

// Activates knockout.js
a=new AppViewModel();
ko.applyBindings(a);

$("#go").click(
    function() {
        $("#txt").text(a.firstName()+" "+a.lastName());
        alert(a.lName());
    }
);

PyQt: change table data => refresh headers

        #change table data => refresh headers

        self.connect(self.Station, QtCore.SIGNAL("cellPressed(int,int)"), self.updateHeader)

        self.connect(self.Train, QtCore.SIGNAL("cellPressed(int,int)"), self.updateHeader)

Python: neo4j.py embedded install

1) install
sudo easy_install neo4j-embedded

2) set path to JVM:

export JAVA_HOME=/usr/lib/jvm/java-6-openjdk/jre

or set path in jpype (/usr/lib/python2.7/dist-packages/jpype/_linux.py):

in string 53: "java_home = os.getenv('JAVA_HOME')" => "java_home = '/usr/lib/jvm/java-6-openjdk/jre/'"

3) in file /usr/local/lib/python2.7/dist-packages/neo4j_embedded-1.6-py2.7.egg/neo4j/_backend.py

string 132 correct follow:

+    jvm_path = get_jvm_path()
	133	+    jvm_args = get_jvm_args()
	134	+
132	135	try:
133		-      jpype.startJVM(get_jvm_path(), *get_jvm_args())
	136	+      jpype.startJVM(jvm_path, *jvm_args)

4) try import neo4j:

>>import neo4j

>>

5) create DB in Python:

from neo4j import GraphDatabase

# Create db

db = GraphDatabase('/data/neo4j_DB')

...

print('all ok')

# Shut down db

db.shutdown()

 

more: http://docs.neo4j.org/chunked/1.5-SNAPSHOT/python-embedded-installation.html

problem with easy_install mysql-python

http://stackoverflow.com/questions/1511661/virtualenv-mysql-python-pip-anyone-know-how

You can install "libmysqlclient-dev" from ubuntu software center

insert new queue in CELERY

CELERY_IMPORTS = ("test1.tasks","new.tasks")

CELERY_ROUTES = \
{
        "test1":
        {
                "queue": "test1"
        },
        "NEW":
        {
                "queue": "NEWtest"
        }
}
CELERY_QUEUES = \
{
        "default":
        {
                "exchange": "default",
                "binding_key": "default"
        },
        "test1":
        {
                "exchange": "test",
                "exchange_type": "direct",
                "binding_key": "test1"
        },
        "NEWtest":
        {
                "exchange": "test",
                "exchange_type": "direct",
                "binding_key": "test1"
        }

}
CELERY_DEFAULT_QUEUE = "default"

info about EC2 servers

get info about ec2 servers with boto:

import boto
from boto import ec2

def getEc2Hosts():
    ec2Hosts = set()
    AccessKey="""AccessKey"""
    SecretAccessKey="""SecretAccessKey"""
    # make connection

    conn = ec2.connect_to_region('eu-west-1', aws_access_key_id = AccessKey, aws_secret_access_key = SecretAccessKey, )

    # get servers status
    instanceStatuses = conn.get_all_instance_status()

    # get info about all servers
    allInst = conn.get_all_instances()

    

    # for our  - only ID, name and system status of the each server:
    instStatus = {i1.id: (i1.state_name, str(i1.system_status)) for i1 in instanceStatuses}

    for i in allInst:
        for g in i.groups:

            # only servers for foto
            if g.name == 'foto-servers':

                #we can use only running servers with OK status
                if instStatus[i.instances[0].id][0] == 'running' and instStatus[i.instances[0].id][1] == 'Status:ok':

                    # add public name to set
                    ec2Hosts.add(i.instances[0].public_dns_name)

    return ec2Hosts

mysqldump: dump by table

dump for one table:
mysqldump -u user -p password databasename table1 > ~/dumpfile.sql

dump for few tables (table1, table2, table3):
mysqldump -u user -p password databasename table1 table2 table3 > ~/dumpfile.sql

mysql: current table encoding

SELECT CCSA.character_set_name FROM information_schema.`TABLES` T,

       information_schema.`COLLATION_CHARACTER_SET_APPLICABILITY` CCSA

WHERE CCSA.collation_name = T.table_collation

  AND T.table_schema = "schemaname"

  AND T.table_name = "tablename";

site for search routes

http://findthebestroute.com - strange site (solve Dijkstra task for few point?)

Python: use Psyco in PyQt4 thread

import psyco

#make function bind in thread before call it

psyco.bind(myFunc())

myFunc()

psyco.bind()

psyco.bind() for ezch function (for all don't working for me):

psyco.bind(mySolve.startSolve)

result=mySolve.startSolve(xArray)

cleanup SVN-files

I copied folder from SVN. And it include svn-files. I deleted it with follow script:

import os

from subprocess import call

d = os.walk('my directory with SVN-tails')

for i in d:

print i # about folder

if i[0][-4:] == '.svn':

print i[0]

# delete SVN-files

call(['rm', '-R', i[0]])

Python: list of the equal lists

 >>> months = [["0"*31]*12]

>>> months
[['0000000000000000000000000000000', '0000000000000000000000000000000', '0000000000000000000000000000000', '0000000000000000000000000000000', '0000000000000000000000000000000', '0000000000000000000000000000000', '0000000000000000000000000000000', '0000000000000000000000000000000', '0000000000000000000000000000000', '0000000000000000000000000000000', '0000000000000000000000000000000', '0000000000000000000000000000000']]

Supervisor: first step

Supervisor  - for manage processes.
 

1) installing:

sudo easy_install supervisor

2) generate config file:

echo_supervisord_conf | sudo tee /etc/supervisord.conf

3) turn on Web-interface: in file /etc/supervisord.conf must have:

[inet_http_server]
port=127.0.0.1:9001

cancel autohide launcher panel on Ubuntu Unity 2D

I take this from gere: http://pingvinus.ru/note/unity-2d-launcher-autohide

1) installing dconf:

sudo apt-get install dconf-tools

2) list of available options:

dconf list /com/canonical/unity -2d/launcher/

in result:

hide-mode
super-key-enable
use-strut

3) always show panel (set use-strut=true):

dconf write /com/canonical/unity-2d/launcher/use-strut true

4) logout - login for show panel

---

restore autohide:

dconf write /com/canonical/unity-2d/launcher/use-strut 0

ssh: too many permissions

There are too many permissions for ssh.

Run command: chmod 600 <keys-files>

some CELERY parameters

# all in UTC :
CELERY_ENABLE_UTC = True
CELERY_TIMEZONE = 'Etc/UTC'

# tasks from and tasks to may be different addresses:
# tasks from
BROKER_URL = "redis://myserver.ru:6379//"

#  tasks to
CELERY_RESULT_BACKEND = "redis"
CELERY_REDIS_HOST = "myserver.ru"
CELERY_REDIS_PORT = 6379

mysql: Got a packet bigger than 'max_allowed_packet' bytes

Mysql get follow message on the import dump: "Got a packet bigger than 'max_allowed_packet' bytes"

Run mysql in the other bash, writing: "set max_allowed_packet=100M"

May be, this parameter also can help: net_buffer_length=1000000

Python: local time for current timezone with pytz

import pytz

tzkiev = pytz.timezone('Europe/Kiev')

tzkiev.fromutc(< UTC time >)

<in result - local time>

transfer data between threads in PyQt

I create signal in start function (in the other words - "wait for signal"):

self.connect(self.s1,QtCore.SIGNAL('printStat(PyQt_PyObject)'),self.printStat)

s1 - child thread;
printStat(PyQt_PyObject) - signal name, which send from thread s1 to main thread;
self.printStat  - function for our reaction by signal ( self = QMainWindow() ).

I create sending signal in child thread:

self.emit(QtCore.SIGNAL('printStat(PyQt_PyObject)'),[list of transfer vars])

I can transfer any PyQt- or/and Python-objects.

Function printStat must get transfer objects:

    def printStat(self, [list of transfer vars]):

show SVG via QGraphicsView

in class QMainWindow we create QGraphicsScene on object myGraphicsView:

self.sc=QtGui.QGraphicsScene(self.myGraphicsView)

In function for show SVG, we write:

r=QtSvg.QSvgRenderer()
r.load(QtCore.QByteArray(mySVG))
r.setViewBox(QtCore.QRectF(0.0, 0.0, 3500.0, 3500.0))
item=QtSvg.QGraphicsSvgItem()
item.setSharedRenderer(r)
item.setScale(20.0)
self.sc.clear()
self.sc.setSceneRect(QtCore.QRectF(0.0, 0.0, 2000.0, 1000.0))
item.setPos(QtCore.QPointF(10,50))
self.sc.addItem(item)
self.graphicsView.setScene(self.sc)
self.graphicsView.centerOn(0,0)

mySVG - source of the SVG-file (as text)

Picture size is 3500 by 3500 pixels.

Threads in PyQt4

I want to create threads in PyQt4 (for normal use GUI).

Make thread class:

class solveThread(QtCore.QThread):
        def run(self):
              <what do you want doing>

Check for use thread with same name and kill it if yes:

        if (hasattr(self,'s1')):
            del(self.s1)
        self.s1=solveThread()


If I want terminate thread by button, I use variable self.isExit=0 in infinite while loop (I don't use Thread.terminate() ). And I use refresh GUI each 5-10 sec (otherwise my GIU is freezing).

time analysis execute Python script

follow command:
python -m cProfile -s time myProject.py

In result: execute time for each function, number of call each function

IMHO, if we want see execute time and calls of only one function, we can call only this function.

wait for press Enter key in console (in Python)

raw_input()

Python: coding in console, which make in cx_Freeze

I make console application in cx_Freeze. And I have follow error with unicode:

Traceback (most recent call last):

  File "C:\Python27\lib\site-packages\cx_Freeze\initscripts\Console.py", line 27

, in <module>

    exec code in m.__dict__

  File "carsDistrib.py", line 117, in <module>

UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-12: ordi

nal not in range(128)

# fix by set default coding:

try:

    sys.setdefaultencoding("

cp1251")

except AttributeError:

    pass

Qt: use Layout for tab in Designer

by follow steps:
- create tab
- create Layout on this tab
- delete Layout from tree of objects
- property Layout must save for this tab

On the video:
http://www.youtube.com/watch?v=8JYEdXDhrTY

PyQt4: python's class from ui-file

I create python's class always after changed of my GUI:
pyuic4 MainWindow.ui -o UIclass.py

where MainWindow.ui - file of QtDesigner, UIclass.py - python's class.

PyQt4: connect

Task: run function mySlotFunction on changing value in combobox:

There is bad idea:
self.connect(self.myComboBox, QtCore.SIGNAL("currentIndexChanged(-1)"), self.
mySlotFunction)  

There is good idea:
self.connect(self.myComboBox, QtCore.SIGNAL("currentIndexChanged(int)"), self.mySlotFunction)   

PyQt: change table headers

We can use table headers with follow:

horizontalHeaderItem ( int column ) const
verticalHeaderItem ( int column ) const

PyQt: how to track change selected cell

#if changed selected cell we print number of current row:
self.connect(self.myTableWidget, QtCore.SIGNAL("itemSelectionChanged()"), self.printCurrRow)

Python to EXE: cx_Freeze is work!

1) on windows installing cx_Freeze.

2) D:\MyProject>C:\Python27\Scripts\cxfreeze myProject_main.py --target-dir folderForExe

myProject_main.py - main file of the project (it's usually run by Python)

folderForExe - target folder for EXE and all libraries.

Python: sorting of the "table" (list of lists)

There is list like "list of the lists": [[1,2,3],[4,5,6],[7,8,9],[11,

12,13],[14,15,16]]
We can sort it by follow script:

import operator
...
myList.sort(key=operator.itemgetter(3))

 

If we can see cortege of number of "columns" in itemgetter (for example: (0,1)), there is sorting order in this cortege.

Django: two buttons in the one form

There are two buttons in the one form:
<form ...>
...

<input type="submit" name=b1 value="Request">
<input type="submit" name=b2 value="Refresh">
</form>

code in views.py:
if request.GET.get('b2',None)!=

None:
   (code for refresh form)
else:
   (code for request data)

Django: using form in views

Some way to use forms in views:
blank form or form with some content:
if request.method=='GET':
   form1=SelectForm(request.GET)
else:
   form1=SelectForm()

And than we send to form necessary corteges for "select" elements (each cortege include  few two-elements cortege and like as ((,),(,),(,),...,(,)) ):
form1.fields['select1'].

choices=((1,'First'),(2,'Second'),(3,'Third'))

We can get data from form:
datas=[]
if form1.is_valid():
   datas=form1.cleaned_data
   dd=datas['select2']

( format of the value "cleaned_data" is dict() )

wsgi installing on apache error

Wsgi and python2.6 see to install in update manager, but error during installation.

Follow command help us:
sudo apt-get install -f

I installed for use with Oracle.

django.forms documentation

Necessary fields:
https://docs.djangoproject.com/en/1.3/ref/forms/fields/

Tehere are few properties for each field:
1) Label - set normal text prefix for field
2) Initial  - set initial values, which must be correcting (initial values handle as unfilled)

3) widget define interface of the field:
https://docs.djangoproject.com/en/1.3/ref/forms/widgets/
                 widget.attrs - is way to do different fields
4) helptext - pop-up text(?)

About forms in docs:
http://docs.djangoproject.com/en/1.3/topics/forms/

Error while installing PostgreSQL

While installing PostreSQL I have follow start DB-server error:
/usr/local/pgsql/bin/postgres -D /usr/local/pgsql/data >logfile 2>&1 &
(errors: LOG:  could not bind IPv4 socket: address alredy in use
HINT:  Is another postmaster already running on port 5432? If not,
wait a few seconds and retry.)

I stop already starting db:
etc/init.d/postresql stop
And all is OK.

There is small test for correct work of PostgreSQL:
/usr/local/pgsql/bin/createdb test
/usr/local/pgsql/bin/psql test

Django: utf-8

Python 2.7, Django 1.3, Apache2

For all to work via utf-8:

1) In the template base.html on the begin of the file we must write:
<meta http-equiv="Cintent-Type" content="text/html" charset=utf-8" Content-Transfer-Encoding: base64>

2) each python file must starting with follow string:
# -*- coding: utf-8 -*-

3) in functions view.py we change request encoding to utf-8:
request.encoding='utf-8'

4) in settings.py we must have:
LANGUAGE_CODE = 'ru'

In django.forms we can use IDs for work with combobox (there is not utf-8 in address string).

Some steps for settings PostgreSQL

There are few steps for setting PostgreSQL:
0) pg_config --version

 - what version are installed?
1) pg_config --configure

  - see server configuration
2) sudo useradd postgres

 - command for add user
3) Make DB dir and delegate permissions to new user:
    mkdir /usr/local/pgsql/data
    chown postgres /usr/local/pgsql/data
4) starting DB:
    $ sudo - postgres
    pg$
5) Inizialize DB:
    pg$ /usr/local/pgsql/bin/initdb -D /usr/local/pgsql/data
6) Starting DB again:
    /usr/local/pgsql/bin/postmaster -D /usr/local/pgsql/data

or 

   /usr/local/pgsql/bin/pg_ctl -D /usr/local/pgsql/data -l logfile start

7) permissions for remote host (not localhost) - in file /usr/local/pgsql/data/pg_hba.conf

in format: host - DB name - user - IP-address - mask - authentication (md5/trust)
  We must also write variable `listen_adresses="*"` in file postgresql.conf
8) start, stop and restart DB:
    pg_ctl start
    pg_ctl_stop
    pg_ctl_restart
9) test: connect to template "template1":
    psql -h 192.168.0.111 -d template1
10) reboot server
11) Then we must see for starting postmaster:
    ps -el | grep post

12) create DB:

    create_db ******
13) create user:

    CREATE USER myuser WITH password 'mypsw';