Monthly Archives: October 2013

Datagrepper installation: exhausting but interesting!


In last blog i have introduced you all about the OPW and the project i am interested on. Moving ahead today i will describe my installation process.
 
Firstly, I visited to the github profile of fedora-infrastructure and cloned the project in my local repository. Thereafter I followed the installation steps given on this link. But at various stages i found errors yelling out on my screen so i thought let describe each steps that may not be listed on official installation procedure but might be helpful.

Installation

1. Open up a terminal window.
2. Install virtualenv and  virtualenwrapper with this command. This will provide you to create and work on a separate environment.

$ sudo pip install virtualenv
$ sudo pip install virtualenwrapper

 Configure the PATH for virtualenvwrapper. To do that, append following lines to your .bashrc file

export WORKON_HOME=/home/charul/.virtualenvs;
source /usr/bin/virtualenvwrapper.sh;

Reload your profile

$ source ~/.bashrc

3. Issue this command to create and then enter the virtual environment

$ mkvirtualenv datagrepper 
$ workon datagrepper        

btw, if you wants to get out of the virtualenv, then

$ deactivate

4. Install dependencies in datagrepper virtualenv

$ pip install -r requirements.txt
$ pip install psycopg2 

If you get error like this

 error: command 'gcc' failed with exit status 1.

Then,


$ sudo yum install python-devel

If it yells error for this dependency #include<libpq-fe.h> requirement, then install that using command:

$ yum install /usr/include/libpq-fe.h

6. Download the rpm file postgresql-9.2.4-7.fc19.src.rpm and install it.

7. Then, issue this command:

 $ pip install .

8. Refer link  — https://github.com/fedora-infra/datanommer/tree/develop/datanommer.models/alembic. 

 $ git clone https://github.com/fedora-infra/datanommer/
 $ cd datanommer.models 
 $ alembic current                                           

Now you have the cloned the datanommer project; copy the datanommer.models folder into the folder where you datagrepper project is there.

 $ cd /datagrepper/datanommer.models
 $ python setup.py install  


Go to alembic.ini and change sqlalchemy.url to sqlalchemy.url
=postgresql+psycopg2://datanommer:bunbunbun@localhost:5432/data     
                       
Then run these commands: 

 $ alembic stamp 2affa1daa804 
 $ alembic upgrade head 

9. Install postgres:

 $ rm -rf /var/lib/pgsql
 $ sudo yum install -y postgresql-server fedmsg
 $ sudo postgresql-setup initdb

10. Edit the /var/lib/pgsql/data/pg_hba.conf. You might find a line like this:
host       all              all          127.0.0.1/32               ident Change it to:
host       all              all          127.0.0.1/32               md5

11. Then, start up postgres

 $ sudo systemctl restart postgresql.service   

12. Become the postgres user (again) and run the psql command.            

 $ sudo su - postgres
 $ psql
 # create database datanommer;
 # create user datanommer with password 'bunbunbun';
 # grant all privileges on database datanommer to datanommer;
 # \q

13. Back in the bash shell (but still as the postgres user), grab a DB dump and restore it:  

 $ wget http://ianweller.fedorapeople.org/datanommer-2013-04-10-0.5.0.dump.xz
 $ xzcat datanommer-2013-04-10-0.5.0.dump.xz | psql datanommer

14. Edit fedmsg.d/example-datagrepper.py and give it these contents

  config = {
    'datanommer.enabled': False,
    'datanommer.sqlalchemy.url': 'postgresql+psycopg2://datanommer:bunbunbun@localhost:5432/datanommer',
    'fedmsg.consumers.datagrepper-runner.enabled': True,
  }

15. As your normal old user self, run the development server:

  $ workon datagrepper
  $ python runserver.py

16.  In a browser, visit http://localhost:5000 to see the output.

17. First execute this command: 

$ sudo yum install -y httpie.
$ http get https://apps.fedoraproject.org/datagrepper/raw/delta==172800.

If its gives you internal server error due to proxy; for that open /etc/bashrc and add this command

export NO_PROXY=localhost,127.0.0.0/8,127.0.1.1
$ source /etc/bashrc

If you want to kill the running server the just press these keys

CTRL-Z 

Also, to kill all programs listening to port(port:5000), so that if you start it next time it will not yell out error

kill -9 `fuser -n tcp 5000` 
At last it took two days to install. In this process i learnt many things. I have gone through various python packages (virtualenv, flask, jinja2, SQLAlchemy etc) and their documentation to understand more about them. Virtualenv is really a handy tool for software developemnt and flask: best tool for portable web-development.
 
I also want to thank my mentor Ralph Bean and Pierre-Yves. They have helped me a lot during the installation process.
 
Thanks!
Advertisements

Outreach Program for Women (OPW 2013)

Hi ladies,

        A good news for you all. I am very happy to say that Gnome has launched its Round Seven of Outreach Program for Women (OPW). The program is for ladies only. So many of you might be wondering: Hey what is this OPW? ladies don’t be so impatient, i am answering it but first see here a beautiful info-graphic about the program. It will give you overall picture of the program. There are various FOSS organization participating in it and they have cool projects for all of us. This year participating organisations list can be found here.
 
 
So hurry up find your interesting projects and start contributing to amazing projects. OK now i will tell you little bit about my story :). After going through the project list of various organisation I found Fedora Infrastructure Data Visualization: Datagrepper / Dataviewer project very exciting as well as challenging. I have selected it because there are many aspects in this project that aligned with my interests i.e. data-visualization, databases, web-technology and best part “working in Python Environment”. Also Fedora is my favorite linux-distro and contributing it means a lot to me. I also contacted my mentor Ralph Bean through email as well as irc chat. He is very supportive kind of person and getting good responses from him.
Now i am moving forward for making some contribution to the project. I have started tweaking API, dependencies etc.
I will keep updating you all with my new journey for making contribution to open source community. If you have any questions, feel free to ask. I will be very happy to answer your questions.

Stay tuned to my blog, till then happy coding.

Cheers !