I’m a cleanliness freak – to the extent of cleaning up my Facebook wall. Over the last week, there’s been a lot of spam with wall messages that read: “Hahaha! Mine is hilarious. Check out yours”? Familiar? Yes, that silly app which tells you how you would look in the future. I never tried it, but tired of the requests, I urged friends to avoid it via a FB update. 13 people liked it, but I only hope its the latter of these 2 messages it conveys: how awful I look, and how useless that app is. (I take everybody’s privacy too seriously, hence some masking)
I had done some research on this topic and found that some script was available in Python. With no knowledge of Python execution, I sought help from stackoverflow & got the answer there pointed to the same Python script 🙁 I decided I had to do something on my own! Ensuring the right no of spaces in the YAML could be a real pain – I haven’t found the error messages to be very friendly.
My challenge was to be able to generate a YAML for a table/schema using the nested-set behaviour in Doctrine. Instead of losing my mind on devising a human-readable format, I reverse engineered. I checked the resulting schema in the database and used that as the base for my CSV. Following this I wrote a simple macro that will output in the required format. This has evolved over a week or so, and has been thoroughly tested. I am sharing this file with 2 examples: one for the JobeetJob schema & other from the Hierarchical Data page on Doctrine.
My previous article The Need to Log & Retain Activity Data argued the very need of logging & retaining data. In this post, I am listing out various logging strategies along with some brief explanation, utility, associated constraints and effectiveness of each method. As highlighted before, most people fail to understand the difference between logs/traces, audit trails and database time-stamps. Each of Log/Trace, Audit Trail & Timestamping has its purpose, pros and cons.
Log or Trace
When I think of a log, the first thing that comes to my mind is a trace consisting of developer injected SOPs (SysOuts), messages/exceptions generated by the server or any third-party component used. This trace could be written to a flat file or a database table.
In the current age of On-Demand & SaaS combined with multi-tenant hosting, we are likely to generate tons of activity data every hour. For this data to be useful to administration & support teams, IT has to plan for its conversion to information. The strategy to implement information logging should be built right into the development process.
However, to most people, that I have communicated with while developing systems,
the terms Audit log, server log, audit trail, time-stamping, change history are synonymous
implementing ‘soft-delete’ probably appears a development overhead
I don’t know if it is because of exposure to ERP or otherwise, but unlike these people, I am overly sensitive to recording audit trails. Are you one of these? Are you not convinced about implementing a logging strategy? Then this post was written thinking about you. Continue reading The Need to Log & Retain Activity Data→
It is the operational crises that results when the warehouse flaunts data abundance, and yet adds no values to business. The crises is also termed data-in-jail (DIJ). This is mostly on account of poor availability or presentation of data. Continue reading Jargon: Data Jailhouse→
A runaway query is a query whose execution time is taking longer than the execution time estimated by the optimizer. Runaway queries can lead to excessive consumption of processing power and yeild nothing.
GIGO or Garbage In Garbage Out, in computing, represents erroneous data entered into systems results in erroneous output. In other senses, it means bad result with bad input. In systems, it is often used to influence management decisions to consciously re-enter data while migrating to a new system, rather than dumping from the legacy system.