Category Archives: Data & Information

Why ignorance is bliss today & 5 ways to ignore online junk

I’m a cleanliness freak – to the extent of cleaning up my Facebook wall. Over the last week, there’s been a lot of spam with wall messages that read: “Hahaha! Mine is hilarious. Check out yours”? Familiar? Yes, that silly app which tells you how you would look in the future. I never tried it, but tired of the requests, I urged friends to avoid it via a FB update. 13 people liked it, but I only hope its the latter of these 2 messages it conveys: how awful I look, and how useless that app is. (I take everybody’s privacy too seriously, hence some masking)

My Status Update
Still wondering what 13 people really liked?

Publicity fail

That application – whatever its name is – is a complete fail in terms of its messaging. Understand the scenario: that app is sending me a persuading message to try it on a friend’s recommendation. The least I expect is Continue reading Why ignorance is bliss today & 5 ways to ignore online junk

Converting CSV/Excel data to Doctrine YAML fixtures

I had done some research on this topic and found that some script was available in Python. With no knowledge of Python execution, I sought help from stackoverflow & got the answer there pointed to the same Python script šŸ™ I decided I had to do something on my own! Ensuring the right no of spaces in the YAML could be a real pain – I haven’t found the error messages to be very friendly.

My challenge was to be able to generate a YAML for a table/schema using the nested-set behaviour in Doctrine. Instead of losing my mind on devising a human-readable format, I reverse engineered. I checked the resulting schema in the database and used that as the base for my CSV. Following this I wrote a simple macro that will output in the required format. This has evolved over a week or so, and has been thoroughly tested. I am sharing this file with 2 examples: one for the JobeetJob schema & other from the Hierarchical Data page on Doctrine.

Let me know if this helps! Vote for the answer on StackOverflow.

To start using:

  1. Download the ZIP file CSV/Excel to YAML Generator v2 & unzip it.
  2. Change the model name Continue reading Converting CSV/Excel data to Doctrine YAML fixtures

Strategies to Log & Retain Activity Data

My previous article The Need to Log & Retain Activity Data argued the very need of logging & retaining data. In this post, I am listing out various logging strategies along with some brief explanation, utility, associated constraints and effectiveness of each method. As highlighted before, most people fail to understand the difference between logs/traces, audit trails and database time-stamps. Each of Log/Trace, Audit Trail & Timestamping has its purpose, pros and cons.

Log or Trace

When I think of a log, the first thing that comes to my mind is a trace consisting of developer injected SOPs (SysOuts), messages/exceptions generated by the server or any third-party component used. This trace could be written to a flat file or a database table.

Example:

2009-31-12 23:59:59 ::: LoginServlet >>> john.doe >>> Incorrect Password
192.168.10.101 – 10/Nov/09:13:55:36 -0700 “GET /logo.gif HTTP/1.0” 200 2326
instantiated Bean: com.detangle.ejbs.whatever
Java.Lang.NullPointerException at ā€¦..
Connected to ProductionDB: Saved record #862
Executed Query: INSERT INTO SUPPLIERSā€¦ : 1 row affected
inside getSuppliersForCategory: Category = “Laptops” Continue reading Strategies to Log & Retain Activity Data

The Need to Log & Retain Activity Data

In the current age of On-Demand & SaaS combined with multi-tenant hosting, we are likely to generate tons of activity data every hour. For this data to be useful to administration & support teams, IT has to plan for its conversion to information. The strategy to implement information logging should be built right into the development process.

The Confusion

However, to most people, that I have communicated with while developing systems,

  • the terms Audit log, server log, audit trail, time-stamping, change history are synonymous
  • implementing ‘soft-delete’ probably appears a development overhead

I don’t know if it is because of exposure to ERP or otherwise, but unlike these people, I am overly sensitive to recording audit trails. Are you one of these? Are you not convinced about implementing a logging strategy? Then this post was written thinking about you. Continue reading The Need to Log & Retain Activity Data