Fedora-Packages: My new coding adventure world!

I have started working on the fedora packages App which is built on the top of the python-turbo gear framework. So this was a new framework to me but in past i had worked on Python-Flask and Rails that helped me to pick up turbogear in good pace.

First question would be why python-turbogear? I would like to give the answer from the official documentation and will help some of it which i have found quite interesting: because

  • Real multi-database support
  • Support for Horizontal data partitioning (aka, sharding)
  • Support for a variety of JavaScript toolkits, and new widget system to make building ajax heavy apps easier
  • Support for multiple data-exchange formats
  • Built in extensibility via standard WSGI components
  • Programmer friendly template system that also works for designers

The task was to “Embed the Datagrepper html messages into the fedora-packages overview tab”. Initially when i was looking through the project and found that it would be challenging to do the task as i was a newbie to the framework and also didn’t have any idea where to start from. So with my past experience i started with small means followed the documentation and built a “hello world” sample App. I started tweaking things into it and finally after 6 hours of struggle i got the turbogear things into my head.

Now i started working on the fedora packages app and added the link to the message card in the search result. When i showed this to my mentor he suggested me to link it into the overview tab of a package and the file which needed to be changes was search_results.mak, overview.py and details.mak.

I have added these lines of code.

  • ¬†details.mak
<div class="history-block">
    <div class="history-cards">
    <a href="${w.history | n}"> Message Cards </a>
  • overview.py
def history(self):
    # html cards link of datagrepper
    datagrepper_link = "http://localhost:5000/raw?order=desc&rows_per_page=5&package="
    pkg_name = self.kwds['package_name']
    return str(datagrepper_link + pkg_name)

Then instead of the link to messages i had to embed the messages in the overview tab.

These were the final changes i made

  • overview.py
def history(self):
    url = tg.config.get('datagrepper_url')
    package = str(self.kwds['package_name'])
    headers = dict(accept='text/html')
    params = dict(order='desc', rows_per_page=5, package=package, chrome='false')
    response = requests.get(url, headers=headers, params=params)
    return {'status_code':response.status_code, 'headers': response.headers, 'text': response.text}

The History messages were supposed to look like the Recent events on this link.Also we need to make things look better and So i have worked on CSS to improve that.

I added the following css

/* history cards */

.message-card {
min-height: 52px;

.history-cards img {
height: 25px;
width: 20px;
margin: 2px auto;

.history-cards p {
display: inline;

Finally the apps looked like this.

Screenshot from 2014-02-06 17:32:00


Fun with kids at Webmakers Kitchen Party

Last month, I attended Webmaker Kitchen Party organized by one of the Mozilla ReMo Shahid Farooqui at his home in Allahabad. I enjoyed it mainly because it was organized for school kids, so everything was like fun doing. The main objective of this event was to make these kids aware about various products of Mozilla, how they can use them and how they can contribute to the community.
The event began with the introduction given by Shahid Farooqui. He told about his role in the community, various events organized by Mozilla and also gave brief introduction about various mozilla products like Webmaker, Firefox, Persona, Firefox add-ons(firebug) and also Firefox OS mobile phone. Then RMA Rahul Ranjan told about FSA program and how to join Mozilla community. After this we volunteers gave small introduction about computer, its parts, uses, internet and e-mail. 12037034126_b54ae2baa9_o

Then comes the best part: drawing making competition because the kids participated so enthusiastically and drew beautiful logos of Firefox and webmaker. The best two drawings were awarded with prizes. It was also a great meeting with Shahid Farooqui as i get to know lot about FOSS, conferences, Mozilla, FSA and WOMOS program.
I also got cool Mozilla stickers and Mozilla t-shirt. ūüėÄ


Group pic

Overall it was a wonderful day!!

Git branch is as cool as tree branch

While working on my next task that was to integrate messages from datagrepper into fedora packages, I learnt few cool stuff about github. I had to change the styling of the datagrepper messages. So, Ralph asked me to make changes and commits in a new branch rather than the develop branch in which I was working before.
If we do not want to make any changes in our master branch (i.e. develop), we can create a new branch that will be a copy of the master branch, then make changes to it and create pull request on this branch. Also for the services that are already working in production, we work on branches only.

How to create a new branch?

I created a branch named fedpkg

Create a new branch on your local machine:

$ git branch fedpkg

Push the branch on github:

$ git push origin fedpkg

Then switch to your new branch:

$ git checkout fedpkg

To see all the branches that have been created:

[charul@localhost datagrepper]$ git branch
* develop

To remove a local branch from your machine:

$ git branch -d the_local_branch

To delete the branch on github:

$ git push origin :fedpkg

To add a new remote to your branch:
(I have added a new remote named upstream)

$ git remote add upstream https://github.com/fedora-infra/datagrepper

To update the local repository with the github repository:

$ git fetch upstream
$ git merge upstream/master

Push changes from your commit into your branch:

$ git push origin upstream

To check the remotes you have:

(datagrepper)[charul@localhost datagrepper]$ git remote -v
origin    https://github.com/charulagrl/datagrepper (fetch)
origin    https://github.com/charulagrl/datagrepper (push)
upstream    https://github.com/fedora-infra/datagrepper (fetch)
upstream    https://github.com/fedora-infra/datagrepper (push)

Added an odometer

My next task was to throw an odometer on front page¬†i.e. on ‘/ ‘ endpoint. This odometer shows the message count i.e the total number of messages and also updates itself with a new incoming message. ¬†Initially i didn’t knew about the about the results but they were really fascinating.

I first added the following code to the ‘/ ‘ endpoint ¬†to get the value of the message-count and render it with index.html.

def index():
    total = dm.Message.grep()[0]
    return flask.render_template('index.html',    docs=load_docs(flask.request), total=total)

Next was to add an odometer¬†, a Javascript and CSS library that can be used for smoothly transitioning numbers. I used it to display message-count so that it can be updated with a new message. It can be done by adding the javascript¬†and a CSS file which can be selected from the following themes.¬†Well to have a glimpse of what odometer is, there is a demo page that you’ll find very interesting. Do have a look at it.

I added the following code to  index.html. I have set the starting value of message count to 123456 and then it will upgrade itself to the value of total.

<h1> <div id="odometer" class="odometer">123456</div>
            odometer.innerHTML = {{total}};
        }, 1000);

Here’s the link to the commit i made. Also you can see below how it looks after adding the odometer to it.

Screenshot from 2014-02-06 18:16:46

After this, I want to update this total with any new incoming message. I did this with the help of websockets. Being a novice in javascript, it took me some-time to understand all this but with the help of few tutorials on websockets and a reference to boilerplate of some other app, i could have got through it.

I added the following javascript function

function WebSocketTest()
    if ("WebSocket" in window)
        // Let us open a web socket
        var ws = new WebSocket("wss://hub.fedoraproject.org:9939");

        ws.onopen = function(e)
            // Web Socket is connected, send data using send()
            ws.send(JSON.stringify({topic: '__topic_subscribe__', body: '*'}));
        ws.onmessage = function (evt)
            count = count + 1;
            odometer.innerHTML = count;
        ws.onclose = function(e){socket=null;};
        ws.onerror = function(e){socket=null;};

Indentation in Python; Sometimes it’s creepy

Yesterday, I was working on my next patch i.e. Patch Number 3. While doing that, I came across a very unusual thing with my system that it was showing me indentation error. I made it correctly indented in terminal but it still shows indentation error. When i opened it with gedit it was again not properly indented. So after searching a lot, I came to know about a cool  feature that is autopep8. It automatically indents your code.
here is, how you can install it via pip.
$ pip install --upgrade autopep8

It will auto-format the code not only indentation but also spacing styles. So it makes the python script to conform  PEP8 style guide.

$ autopep8 your_code.py 
# to auto-format your code
After solving this somehow,  i made a new pull request and create a new commit. But then i realized that there were few things that were i need to change. Also, the problem of indentation error was not resolved.
So first thing i had to do was to delete my git commit. I learned few commands that i would like to share with you.
To know which commit is the head pointing to:
$ git reset --hard HEAD~1

The HEAD~1 means the commit before head.
If you want to move you head to some earlier commit,then

$ git log

This will list all the commits made by you along with their commit id. You can find the commit-id of the commit you want to move your head to

$ git reset --hard <sha1-commit-id>

If you want to delete the commit that you have already pushed, then:

$ git push origin HEAD --force
After doing this, I again made commit after making changes in the code. But my problem was still there. There were indentation errors and also trailing white-spaces errors. Then pypingou asked me to make changes in the ~/.vimrc file and it really worked for me.
They are:
set list
set listchars=tab:→\ ,trail:·
This will allow you to quickly check your trailing spaces or when it uses tab/spaces. So, finally my problem was solved ūüôā¬†

Interesting results with fedmsg.meta

The next task my mentor gave me was to convert the messages that were in json dictionary like format into something that is good looking and in human readable form. For that, he asked me to read documentation on fedmsg. Fedmsg is a python package that can be used to send and receive messages to and from applications. After reading it, I found fedmsg.meta module very interesting and something that could solve my problem. It has beautiful functions that can be used to produce nice html and can convert messages into some beautiful string representations, icons, secondary_icon, link, title and subtitle. Also I liked it because it produced interesting results and is easy to learn and use.

I had message in json-dict format

  I wanted to have output like this.

The first thing i do is to convert the messages(dict format) into string.I used

       fedmsg.meta.msg2repr(msg, legacy=False, **config)

You will have to implement this code

>>> import fedmsg.config
>>> import fedmsg.meta
>>> config = fedmsg.config.load_config([], None)
>>> fedmsg.meta.make_processors(**config)
>>> text = fedmsg.meta.msg2repr(msg_in_dict, legacy=False, **config)                        

To get the desired output, I wrote request_wants_html function in this way

# return HTML content else json
if request_wants_html():
# convert string into python dictionary
obj = json.loads(body)
# extract the messages
messageList = obj["raw_messages"]
#using fedmsg.meta function
config = fedmsg.config.load_config([], None)

finalMessageList = []

for msg in messageList:
d = {}
# create primary icon associated with message
icon = fedmsg.meta.msg2repr(msg,legacy=False,**config)
d['icon'] = icon
# create URL associated with message
link = fedmsg.meta.msg2link(msg, legacy=False, **config)
d['link'] = link
# create title associated with message
title = fedmsg.meta.msg2title(msg, legacy=False, **config)
d['title'] = title
# create secondary icon associated with message
secondary_icon = fedmsg.meta.msg2secondary_icon(msg, legacy=False, **config)
d['secondary_icon'] = secondary_icon
subtitle = fedmsg.meta.msg2subtitle(msg, legacy=False, **config)
d['subtitle'] = subtitle


return flask.render_template("raw.html", response=finalMessageList)
To render the title, subtitle, icon, link and secondary_icon, i used the html file this way
{% for dict in response %}
<p><b>{{dict['title']}}</b></p> </br>
<p> {{dict['subtitle']}} </p> </br>
<A HREF={{dict['link']}}><Img SRC= {{dict['icon']}} WIDTH=30 HEIGHT=40></A>
<Img SRC={{dict['secondary_icon']}} WIDTH=30 HEIGHT=40></A>
{% endfor %}

So, this was my patch 2. In my coming blogs I will be sharing with you a short introduction to jinja2 framework.

Happy learning ūüôā

My First Patch: Cool Coding

My first patch was to differentiate between the accept headers using flask’s methods and returns content accordingly. It means that if the accept header is in “application/json” it should return the content in JSON format and in case of “text/html”, it should return “HTML” format and for other cases it should return ‘JSON’ by default.
I was given a¬†link to use it as reference. There is a function request_wants_json that returns true if accept header is “application/json”. otherwise it returns false. By¬†default¬† an HTTP request is rendered as html in browser. But, we can provide accept header on our wish and can give preference to other mimetype. I came to know that there are 17¬†mimetypes in total and many more are coming.

Initially i thought why not check for “text/html” header’s type using flask method i.e.¬†flask.request.headers.get(‘Accept’)¬†and then return content according that. But my mentor suggested not to use that as they wanted codes in more engineered way and extensible one ūüôā

So I moved on the flask accept header snippet and modified ‘request_wants_json’ function to ‘request_wants_html’ so as to put html on¬† higher quality than ‘application/json’ and ‘text/plain’.So, it returns true in case of “text/html” and false otherwise.
def request_wants_html():     
    best = flask.request.accept_mimetypes.best_match(['application/json',\
    return best=='text/html' and flask.request.accept_mimetypes[best]\ 
             > (flask.request.accept_mimetypes['application/json'] or
             \ (flask.request.accept_mimetypes['text/plain']) 
I also made the following changes in the /datagrepper/app.py. It uses request_wants_html to obtain the desired output.
# return HTML content else json
if request_wants_html():         
   return "HTML Format"     
   return flask.Response(response=body,                
It returns the “HTML Format” in case if accept header¬† is “text/html” otherwise it returns the JSON format. Later on i will have to add something in this HTML part to make it return some beautiful HTML content.¬†
This is how I submitted my first patch. It was a nice beginning.