Category Archives: Technology

Converting CSV/Excel data to Doctrine YAML fixtures

I had done some research on this topic and found that some script was available in Python. With no knowledge of Python execution, I sought help from stackoverflow & got the answer there pointed to the same Python script 🙁 I decided I had to do something on my own! Ensuring the right no of spaces in the YAML could be a real pain – I haven’t found the error messages to be very friendly.

My challenge was to be able to generate a YAML for a table/schema using the nested-set behaviour in Doctrine. Instead of losing my mind on devising a human-readable format, I reverse engineered. I checked the resulting schema in the database and used that as the base for my CSV. Following this I wrote a simple macro that will output in the required format. This has evolved over a week or so, and has been thoroughly tested. I am sharing this file with 2 examples: one for the JobeetJob schema & other from the Hierarchical Data page on Doctrine.

Let me know if this helps! Vote for the answer on StackOverflow.

To start using:

  1. Download the ZIP file CSV/Excel to YAML Generator v2 & unzip it.
  2. Change the model name Continue reading Converting CSV/Excel data to Doctrine YAML fixtures

Jargon: Code Profiling

Code profiling is a form of dynamic program (code) analysis to investigate the program’s behavior during execution. It is usually done to determine sections of code that can be optimized – to increase overall speed & reduce resource requirements. Code profiler can come in several varieties – from those that measure the frequency & duration of function calls to others that track memory usage. An emulator for micro-processor code (that simulates the processor’s instruction set) is also a type of profiler that measures behavior for the entire execution cycle – invocation to termination.

I first heard of code profiling in the context of reverse engineering software architectures. Often it may so happen that applications with very little documentation are to be enhanced or re-engineered. One way to understand is by analyzing code, which can be a tedious process. Such static program analysis may yield incomplete understanding as the polymorphic calls can’t be determined until run-time. A code profiler determines polymorphic function calls and identifies overloads used during execution which helps establish relationships between classes & components that can be used to reveal the underlying software architecture, if not for a specific pattern.

Get updates from any web page via a Feed43 RSS feed

As is evident from several posts on my blog, I am a total food freak! I am also a regular burrper. I have written several valuable reviews at Burrp and also received written appreciation from Burrp. A couple of months back I was thinking of copying food review from Burrp to my blog. Since that was going to be tedious, I went looking for a RSS feed from Burrp. It was sad they don’t offer it, but I wrote them my feedback. Now I was left with no option but to think out of the box!

The Need
The Need

And then the Web 2.0 enthusiast in me came to life! I had been using a service called Feed43 to process several feeds and remembered that it allows creating a feed out of any page on the internet. I checked the HTML source code of my Burrp profile page to find that reviews were quite structured in terms of markup. So why not let Feed43 read out the page and create a feed for my reviews? This way I won’t have to copy anything manually. Moreover, whenever I post a review to Burrp, it will be available on my blog in less than 6 hours (that’s the refresh rate for free feeds at Feed43) Continue reading Get updates from any web page via a Feed43 RSS feed

Tarpipe workflows for publishing updates to multiple social media sites

Off-late, I have had too much social presence on the internet. How do I manage it? Ping.fm! This service lets you pre-configure & then simultaneously update multiple social media sites by pinging Ping.fm which can be done via email, SMS (to a UK number – noooooh!) or a Jabber/Gtalk bot. Now that’s enough for the aam zindagi, but when you live the mentos (or should I say prasadgupte) zindagi, you might just want some processing to be done before you post to multiple services. That’s where TarPipe kicks-in! Here is a short tutorial.

Tarpipe lets you build custom workflows through an intuitive UI (like Yahoo pipes) to control how, where, and what part of your data is to be published. In my example, I’m creating a workflow to upload a photo to Facebook via email and then post its URL to Twitter & FriendFeed. I avoided using a URL shortening service to KISS (Keep It Simple, Stupid)

My First TarPipe Workflow
My First TarPipe Workflow

I first drag a Email Decoder connector onto the canvas, and then Facebook, Twitter and FriendFeed. Note that bubble on the left side of a connector indicates ‘input’ & the one on the right side indicates ‘output’.

So when I connect the Mail Attachment bubble to the Photo in Facebook, it means that the attachment (a photo) will act as input to Facebook. On similar lines, the message body acts as the caption for the photo. The URL for the photo, generated by Facebook, will be available as output which I will use as a link in Twitter. The photo-thumbnail goes to Friendfeed along with the title & link. The title in either case comes directly from the email. Continue reading Tarpipe workflows for publishing updates to multiple social media sites

Jargon: Electronic Invoice Presentment and Payment (EIPP)

Electronic Invoice Presentment and Payment, or EIPP, is more general concept (as compared to ERS discussed earlier) that is based on electronic invoice submission. ERS uses Advance Shipping Notices (ASNs) instead of invoice. However, they share benefits: avoidance of data entry, errors and exceptions, lost invoices and vendor inquiries. Most solutions are capable of receiving invoices in CSV, XML and few other formats over EDI.

EIPP or e-Invoicing is part of the larger procure-to-pay (P2P) cycle (will be writing soon on this). A huge challenge lies in supplier on-boarding: getting suppliers to automate at their end. In the current era, supplier enablement can be hastened by hosting an internet facing portal – called Supplier Portal in sourcing jargon – that suppliers can log-on to and key-in their invoices. ‘Flipping-the-PO’ is a standard feature that saves data entry effort for the supplier, and minimizes error. On referencing the PO being invoiced, information about line items viz. description, ordered quantity & price are defaulted. This helps reduce the number of expections in automated invoice matching. Imagine this as handing over your groceries list to a baniya who converts it to a bill by stamping his name and adding prices & total (and discounts if the baniya is willing to spare)

References:
http://scm.ncsu.edu/public/facts/facs041014.html
http://www.agilent.com/oracle_supplier/downloads/ERS_supplier_guide.pdf
http://www.jpmorgan.com/tss/General/Invoice_Management/1159348844579

Jargon: FITALY Keyboard Layout

FITALY Keyboard Layout
FITALY Keyboard Layout

FITALY is a keyboard layout that places the most commonly-used letters closest to the centre, to minimize finger movement while entering a word. Designed by Jean Ichbiah (Patent), it is specifically optimized for stylus or touch-based input. The name, FITALY, is derived from the letters occupying the second row in the layout (like QWERTY comes from the 1st row of standard keyboards)

The aim of the design is to optimize text entry by organizing keys to minimize key-to-key finger movement, allowing faster input through one-finger entry (compared to 10 fingers required to type efficiently on QWERTY layout).  As compared to the 3-row QWERTY keyboard, FITALY has 5 rows with atmost 6 letters in a row (as against 10 on QWERTY).

Letter Frequencies in the English language
Letter Frequencies in the English language

Continue reading Jargon: FITALY Keyboard Layout

Strategies to Log & Retain Activity Data

My previous article The Need to Log & Retain Activity Data argued the very need of logging & retaining data. In this post, I am listing out various logging strategies along with some brief explanation, utility, associated constraints and effectiveness of each method. As highlighted before, most people fail to understand the difference between logs/traces, audit trails and database time-stamps. Each of Log/Trace, Audit Trail & Timestamping has its purpose, pros and cons.

Log or Trace

When I think of a log, the first thing that comes to my mind is a trace consisting of developer injected SOPs (SysOuts), messages/exceptions generated by the server or any third-party component used. This trace could be written to a flat file or a database table.

Example:

2009-31-12 23:59:59 ::: LoginServlet >>> john.doe >>> Incorrect Password
192.168.10.101 – 10/Nov/09:13:55:36 -0700 “GET /logo.gif HTTP/1.0” 200 2326
instantiated Bean: com.detangle.ejbs.whatever
Java.Lang.NullPointerException at …..
Connected to ProductionDB: Saved record #862
Executed Query: INSERT INTO SUPPLIERS… : 1 row affected
inside getSuppliersForCategory: Category = “Laptops” Continue reading Strategies to Log & Retain Activity Data

The Need to Log & Retain Activity Data

In the current age of On-Demand & SaaS combined with multi-tenant hosting, we are likely to generate tons of activity data every hour. For this data to be useful to administration & support teams, IT has to plan for its conversion to information. The strategy to implement information logging should be built right into the development process.

The Confusion

However, to most people, that I have communicated with while developing systems,

  • the terms Audit log, server log, audit trail, time-stamping, change history are synonymous
  • implementing ‘soft-delete’ probably appears a development overhead

I don’t know if it is because of exposure to ERP or otherwise, but unlike these people, I am overly sensitive to recording audit trails. Are you one of these? Are you not convinced about implementing a logging strategy? Then this post was written thinking about you. Continue reading The Need to Log & Retain Activity Data

Amazing Comments In Source Code by Developers

As I was getting over Monday blues, Aditya Tripathi sent this funny-yet-realistic! I’m sure this will get every dev-devil laughing with delight; reminiscing the KLOC written with cryptic comments or nothing at all. Here you go, the best code comments seen in source code…!

//When I wrote this, only God and I understood what I was doing
//Now, God only knows

/*
* You may think you know what the following code does.
* But you dont. Trust me.
* Fiddle with it, and youll spend many a sleepless
* night cursing the moment you thought youd be clever
* enough to “optimize” the code below.
* Now close this file and go play with something else.
*/ Continue reading Amazing Comments In Source Code by Developers