Archive for July, 2007 is fast again, thanks to a new and improved SQL database schema Jul 26 2007

For quite a while, has been slow. Ever since it hit, oh, around 1.4 million items in its database (now up over 2 million) the queries that were responsible for building the home page view had gotten slower and slower.

The old design

The problem was one of normalization, and the desire to not have data going into two places. The design originally worked this way:

  1. Look up the latest posts. Find all the latest ones that have a distinct feed_id. This was running against the feed_data table, which now has over 2 million rows.
  2. Next, look up the feed information for each feed_id found. Originally, I couldn’t figure out a way to do this all in one query. So this was another 10 individual SQL queries, looking up information about the feed needed to display on the home page (like the name of the feed, URL, etc.) This query did a join of the feed_data with the feed_info tables. This is what took progressively longer and longer, the more rows in the feed_data table.
  3. Next, combine the results of finding the 10 most recent posts from the 10 most recently updated feeds into an array using PHP, then pass those results from the Data Access Object (DAO) to the presentation layer for output on the home page.

This ended up being really, really slow on the homepage load (like over 2 minutes). This was obviously too slow. I implemented caching at a number of layers to mitigate the problem, which made for a very speedy second page load, but the first one was still way too slow.

After consulting with my friend Ben, he agreed that it was an inherently expensive query to run. We agreed that it needed to be re-architected to be able to significantly improve performance.

The new design

The new design required creating a new table dedicated to the home page posts where, when a RSS feed is read, it inserts the most recent new item into the new table.

The new process works like this:

  1. RSS feed is read. Any new items are first inserted into the main feed_data table, and the feed_info table is updated as well with a last checked timestamp.
  2. Next, the new table is updated, using a MySQL REPLACE command, with the combined feed information as well as item information
  3. The complicated 11 queries that used to be run are now replaced by a dead-simple single query that will only ever have to search through the total number of feeds in the system (currently 753 rows), instead of the over 2 million rows before.

These new changes have made using much more usable again, and it is way faster. The homepage loads in under a 1/2 second now, just like it should.

Check it out: – Really Simple News For You

Executive Order: Blocking Property of Certain Persons Who Threaten Stabilization Efforts in Iraq Jul 22 2007

More coverage of this here:

This Executive Order, signed by President Bush and released on July 17, 2007, says that if the Secretary of the Treasury decides that you might commit violence against the efforts for Iraq sometime in the future, that they can seize all of your property. It is up to them to decide what constitutes a “significant risk of committing” an offence. Does that seem problematic? Not if you support their efforts. If, however, you decide to protest against what you don’t agree with, or maybe you read web sites (remember that AT&T is watching what you do online for the government) that encourage people to take action against what they consider to be illegal actions taken by this administration, if you can be seen to be in opposition, WHAT IS THERE TO STOP THEM FROM DECIDING THAT YOU POSE A THREAT, and that, to protect the USA, your property should all be taken away, with no recourse?

As I’ve written many times, NOW IS THE TIME TO IMPEACH BUSH AND CHENEY. Write your Congressmen and Representatives, today. Tell them why they’ve got to support impeachment, NOW. It is the law, it is their job. There is nothing more important than acting with integrity on this issue. If we fail, as a nation, to hold our leaders accountable to the rules and laws that we’ve set up, we deserve to get what’s coming to us.

Complete text of the Executive Order

By the authority vested in me as President by the Constitution and the laws of the United States of America, including the International Emergency Economic Powers Act, as amended (50 U.S.C. 1701 et seq.)(IEEPA), the National Emergencies Act (50 U.S.C. 1601 et seq.)(NEA), and section 301 of title 3, United States Code,

I, GEORGE W. BUSH, President of the United States of America, find that, due to the unusual and extraordinary threat to the national security and foreign policy of the United States posed by acts of violence threatening the peace and stability of Iraq and undermining efforts to promote economic reconstruction and political reform in Iraq and to provide humanitarian assistance to the Iraqi people, it is in the interests of the United States to take additional steps with respect to the national emergency declared in Executive Order 13303 of May 22, 2003, and expanded in Executive Order 13315 of August 28, 2003, and relied upon for additional steps taken in Executive Order 13350 of July 29, 2004, and Executive Order 13364 of November 29, 2004. I hereby order:

Section 1. (a) Except to the extent provided in section 203(b)(1), (3), and (4) of IEEPA (50 U.S.C. 1702(b)(1), (3), and (4)), or in regulations, orders, directives, or licenses that may be issued pursuant to this order, and notwithstanding any contract entered into or any license or permit granted prior to the date of this order, all property and interests in property of the following persons, that are in the United States, that hereafter come within the United States, or that are or hereafter come within the possession or control of United States persons, are blocked and may not be transferred, paid, exported, withdrawn, or otherwise dealt in: any person determined by the Secretary of the Treasury, in consultation with the Secretary of State and the Secretary of Defense,

(i) to have committed, or to pose a significant risk of committing, an act or acts of violence that have the purpose or effect of:

(A) threatening the peace or stability of Iraq or the Government of Iraq; or

(B) undermining efforts to promote economic reconstruction and political reform in Iraq or to provide humanitarian assistance to the Iraqi people;

(ii) to have materially assisted, sponsored, or provided financial, material, logistical, or technical support for, or goods or services in support of, such an act or acts of violence or any person whose property and interests in property are blocked pursuant to this order; or

(iii) to be owned or controlled by, or to have acted or purported to act for or on behalf of, directly or indirectly, any person whose property and interests in property are blocked pursuant to this order.

(b) The prohibitions in subsection (a) of this section include, but are not limited to, (i) the making of any contribution or provision of funds, goods, or services by, to, or for the benefit of any person whose property and interests in property are blocked pursuant to this order, and (ii) the receipt of any contribution or provision of funds, goods, or services from any such person.

Sec. 2. (a) Any transaction by a United States person or within the United States that evades or avoids, has the purpose of evading or avoiding, or attempts to violate any of the prohibitions set forth in this order is prohibited.

(b) Any conspiracy formed to violate any of the prohibitions set forth in this order is prohibited.

Sec. 3. For purposes of this order:

(a) the term “person” means an individual or entity;

(b) the term “entity” means a partnership, association, trust, joint venture, corporation, group, subgroup, or other organization; and

(c) the term “United States person” means any United States citizen, permanent resident alien, entity organized under the laws of the United States or any jurisdiction within the United States (including foreign branches), or any person in the United States.

Sec. 4. I hereby determine that the making of donations of the type specified in section 203(b)(2) of IEEPA (50 U.S.C. 1702(b)(2)) by, to, or for the benefit of, any person whose property and interests in property are blocked pursuant to this order would seriously impair my ability to deal with the national emergency declared in Executive Order 13303 and expanded in Executive Order 13315, and I hereby prohibit such donations as provided by section 1 of this order.

Sec. 5. For those persons whose property and interests in property are blocked pursuant to this order who might have a constitutional presence in the United States, I find that, because of the ability to transfer funds or other assets instantaneously, prior notice to such persons of measures to be taken pursuant to this order would render these measures ineffectual. I therefore determine that for these measures to be effective in addressing the national emergency declared in Executive Order 13303 and expanded in Executive Order 13315, there need be no prior notice of a listing or determination made pursuant to section 1(a) of this order.

Sec. 6. The Secretary of the Treasury, in consultation with the Secretary of State and the Secretary of Defense, is hereby authorized to take such actions, including the promulgation of rules and regulations, and to employ all powers granted to the President by IEEPA as may be necessary to carry out the purposes of this order. The Secretary of the Treasury may redelegate any of these functions to other officers and agencies of the United States Government, consistent with applicable law. All agencies of the United States Government are hereby directed to take all appropriate measures within their authority to carry out the provisions of this order and, where appropriate, to advise the Secretary of the Treasury in a timely manner of the measures taken.

Sec. 7. Nothing in this order is intended to affect the continued effectiveness of any rules, regulations, orders, licenses, or other forms of administrative action issued, taken, or continued in effect heretofore or hereafter under 31 C.F.R. chapter V, except as expressly terminated, modified, or suspended by or pursuant to this order.

Sec. 8. This order is not intended to, and does not, create any right, benefit, or privilege, substantive or procedural, enforceable at law or in equity by any party against the United States, its departments, agencies, instrumentalities, or entities, its officers or employees, or any other person.



July 17, 2007.

# # #

From: Executive Order: Blocking Property of Certain Persons Who Threaten Stabilization Efforts in Iraq

An excellent comment from Austin Cooper on TPMmuckraker:

“I used to think like you people… then I grew up.”
— JTho

It isn’t necessary to break out the tinfoil.

The country has, since the Truman administration, had emergency plans for continuity of government and responses to a variety of worst-case natural or man-made disasters — including pandemics, urban insurrections (after the summer of 1966), and, now, terrorist incidents. Insofar as I know, FEMA is supposed to be the principal governmental agency coordinating the execution of these emergency plans.

There’s nothing inherently sinister about this.

Bush and Cheney have done more to inflate the alleged ‘powers’ of the Executive branch, including:

1) Suspension of Habeus Corpus;

2) Establishment of unprecedented surveillance programs, and the expansion of federal law enforcement powers to conduct wiretapping and secretly obtain commercial, financial or other personal records with no true oversight;

3) Development of the concept in law of persons (including U.S. citizens) declared “enemy combattants”, whose legal rights are effectively voided, and who can be detained indefinitely solely upon Executive order;

4) The development of a legal basis for the use of “extreme coercion” — torture — directed by Executive Order;

5) Expanded powers of search and seizure for law enforcement under the Patriot Act;

6) The use of thousands of “signing statements” attached to laws passed by Congress — all essentially stating that execution of these laws are subject to interpretation by the Executive Branch;

7) Restrictions on air travel for thousands of citizens, often erroneously, and expanded restrictions in being granted a Passport; all based upon a series of secret “Watch Lists” with no oversight;

8) The subject of this TPM article — essentially establishing another principle that the Executive branch may order the financial assets of specified groups or individuals (including U.S. citizens) frozen, solely upon Executive order.

All of these are facts. In and of themselves, each point has raised concern and debate — but that debate has been ultimately meaningless because without challenge, the Executive branch continues to do as it wishes.

It is also a fact that Bush and Cheney, and their ‘administration’, have assumed more unilateral power than any other Executive branch in American history, even overshadowing Lincoln’s during the Civil War.

I don’t argue that, with the next terrorist attack in America, we might expect martial law, freezing of assets and mass arrests of whole classes of persons — declared enemies of the state, on the basis of public comments or private communication, or association with other ‘enemies’ — and that tens or hundreds of thousands could be held in secured compounds across the country.

I’m not suggesting that the current ‘administration’ — frightened that any potential criminal activity it has committed might be exposed, and convinced that they are the only persons to save America from internal and external threats — are desperate or fanatical enough to use the cover of a terrorist ‘incident’ to establish an effective dictatorship.

There’s no ‘wink, wink’ here. I’m not arguing that any of this will occur. I mention the secured compounds (they’re real; Haliburton has the contract) not for the tinfoil value, but because their construction hasn’t been adequately explained by anyone, or the expenditure of public funds. However, it’s fair to mention that they could just as easily be used in a variety of scenarios — some benign, some not — calling for the housing of large numbers of refugees, illegal immigrants, quarrantined pandemic flu victims, etc.

I only offer one opinion: The United States has not fought a real war on its own soil since 1865. We’ve never been physically invaded or occupied by a foreign army since 1813.

We have never experienced a true breakdown in government in our lifetimes — or a dictatorship a la Fascist Italy, nazi Germany; Soviet Russia or Communist China; or the worst of the Eastern Bloc.

We haven’t experienced living in fear of a true Secret Police, whose purpose is to enforce laws that protect ideological or religious orthodoxy through torture, imprisonment and ultimately, murder.

Somehow, as Americans, we believe that our system of government (based, supposedly, on the Rule of Law) renders us immune from the repeating patterns of governments and history that mark the Old World. It’s part of our national self-image.

Unless we actively preserve those things we feel protect us from authoritarian rule, from ending up as “just another failed experiment” in the governing of human affairs, then it’s foolish to believe we *are* protected, or special — and that It Can’t Happen Here.

I don’t believe that we are immune from history, or that persons like Bush and Cheney are any less faliable than the rest of us.

The real questions are whether, by continuing to assert the authority of the Executive branch as trumping all others, that Bush, Cheney and their ideological allies are convinced that they alone perceive the ‘real world’ and understand what America is — and, that the current war and expansion of Executive authority are critical necessities to protect “America”;

And, whether they feel that in order to protect “America”, they must maintain their control of government, the military, and foreign policy — *at any cost* — because their concept of The State is indivisible from their persons, and their ideologies.

A nation, finding itself in the hands of leaders at this particular crossroads, isn’t a new occurance in history. It is, however, a new enough occurance for us that we aren’t able to recognize the dangers inherent in the situation.

Posted by: Austin Cooper
Date: July 20, 2007 1:14 PM

I’m going to Video Games Live show in Louisville, KY tomorrow! Jul 20 2007

Video Games Live

I am totally looking forward to this. For years I have been looking for someone to do this exact show, and now someone has gone and done it. I’m very excited. And it’s scheduled to come to Columbia, MO later this year in October. So I may get to see it twice… w00t.

Check it out: Video Games Live

A List Apart: Articles: Conflicting Absolute Positions Jul 17 2007

Nice article on ALA about how to create a layout using almost pure CSS where there is a fixed-width sidebar and a dynamically resizing right-hand main content area without using Javascript to dynamically resize the page.

All right, class. Using CSS, kindly produce a liquid layout that contains a fixed-width, scrolling side panel and a flexible, scrolling main panel. Okay, now do it without JavaScript.

Check it out: A List Apart: Articles: Conflicting Absolute Positions

Web Devout Jul 11 2007

Some excellent articles here about good web building practices.

Welcome to Web Devout! This site is aimed at promoting the health of the Web by providing web developers and the public with the knowledge and tools needed to help the Web move forward.

Check it out: Web Devout

Thanks for the pointer, Ben.

Car Wallpapers Jul 2 2007

Pagani Zondar

Bruce’s project is starting to really flesh out nicely. If you’re looking for some sexy exotic car wallpapers, check out

sell diamonds