Author Archive

Creativity, Opportunity and your Career

Are there times in life when you find that you are more creative than others? Are there certain times during the day when you find that you have a knack or otherwise when your mind is best able to relax and engage in a creative moment?

I find that it’s interesting – this creative process. It does not come about during the normal hustle and bustle of work and life. It is hard to find when one is entwined in day to day life and trying to keep up with a workload or with family, personal interests or otherwise any sort of commitment which requires engagement.

Do you know where that spot is in your life? Do you know how to create that moment? Do you create that moment?

Personally, it’s difficult for me to find the time to engage in this process at a level that I find of interest or where they are productive enough to matter.

I have a routine which I go through and this routine can be accomplished in any sort of environment.

At this moment, I’m in my process of finding my creative space – this takes, for me, several forms.

The first is physical. I need to be comfortable. If I’m hungry, it’s more difficult to relax and focus – if I’m tired, my focus isn’t what it needs to be in order to settle into a constructive form of creative thought.

The second is mental. I need to have things on my mind wrapped up – little things like remembering to forward an email about the landscaping to my wife, or setting up a reminder to call someone back or scheduling the meeting that’s been on my mind, paying a bill, etc… get rid of the ticky-tack stuff in your brain so that it’s not of any concern.

The third is emotional. We are all emotional creatures. As a man, it took me time to realize that I am incredibly emotional and this is an incredible blessing and one which should be embraced. For me, I find that music can often bring me to a an emotional state where I find that my foot is tapping, I have a smile on my face and I’m singing along to some new song I’ve recently heard (even though I’m terrible at using the actual lyrics).

The final item, for me, is environment. Turing off the phone. Closing the door, turning up the music a bit higher, or putting on headphones. I need to be lost in a lot of “white noise” which I’m familiar – in this manner, outside stimuli which my brain isn’t anticipating is out of the picture. The same is true with line of sight. I ensure that nothing is in my line of sight which isn’t expected or anticipated. This includes co-workers, pets, TV, internet, etc… The goal here, is to eliminate the outside stimulus which your brain isn’t already expecting.

Today I find myself in this space, on an airplane. And now you, the reader are saying, are you kidding me? You just broke like five out of the four rules you mentioned above. Heh yes, indeed; I did. The point is, that I actually find myself in a very familiar environment – I typically sit in one of two seats on each flight. So, the noise is about the same, the routines of the flight attendants is about the same, etc…

Today I find myself listening to the Killers and listening to the album “Battle Born”. I just finished my typical fare which consists of the Thai Chicken roll and a club soda with a lime.

The work consists of thinking outside of the box as it relates to technical architecture components but more importantly the relationships, conversation and language around what I’m thinking. You see, so often in life we have a picture of something in our minds eye. Meaning that we “get-it”, we can “see” the answer – yet we struggle with articulating what that means or why it’s important or how do we get there?

These are some pretty important conversations in your job. You have them all of the time. They have been known to frustrate you – perhaps to the extent that they have been known to drive you to monster or dice to look for “another company” which isn’t so “dysfunctional”.

How do I know this you ask? Well, I’ve been there and I’ve done that, more than once. However, it dawned on me one day that something else was going on… you know the definition of insanity is doing the same thing over and over and expecting a different result. Here you are, in the same boat, again, doing what you have always done, because, you are of course, right. But guess what? It doesn’t matter if you are right. It doesn’t matter how wrong everyone else is. What matters is you, because after all, who can you control? Yeah, that’s what I thought. It’s up to you to change the game. The game cannot be changed if you keep doing the same thing over and over, now can it?

It can be tough to look in the mirror and realize that the degree(s), certification(s), successes which you’ve had in your career to date don’t matter all of the time. This is true for your manager, for their manager, etc… its true all the way up the chain.

You see folks, as database people, we are given an opportunity unlike many others. The platform exists, based on what you do, day to day to have an incredible impact to your overall organization. Now, this impact, it can take more than one form. It can be positive. It can be negative. I’ve seen both. I’ve done both. Choosing the positive path is highly recommended. Choosing the positive path is significantly harder. Choosing the positive path requires a bit of humility, grace, patience, perseverance and temperance. As database people, we are often “right” – you see, this is the rub, from an organizational standpoint. If you have *EVER* said “I told you so” or “I warned you…” etc… you have failed yourself and you’ve severely impacted the value that you might be able to have within your organization…. Screech! Say what? – But I was RIGHT, dammit! Heh. Yeah, you were. So what?

Put yourself in the shoes of the person or team you said that to. Seriously. Stop reading. Close your eyes. Think about where they were, what they had just worked on – what are they thinking in that moment? I’ll help you out, they are thinking “wow, what an ass”, or “here we go again…” or “why do they (the company you work for) tolerate this kind of behavior”, etc…

Yup – and if we are all honest, at some point in our jobs / careers, we have thought the same thing of others.

So, what does this all mean, why does it matter, what is the point of this post?

As a data professional we are uniquely positioned to see the “world” from a vantage point that others aren’t privy to. We have the ability to take on responsibilities outside of our specific domain. We have the ability to contribute in areas where many others don’t have an opportunity. Your company can benefit from you not thinking about yourself and others but instead thinking about the challenge at hand. Thinking about the strategic or corporate initiatives and then finding a way to contribute to them. It’s about finding a way to help the development team or the server and san guru’s, or if you are really good, the project management team. It’s easy to help each one of those groups. Take them out to lunch, get to know their story as it relates to work. Ask how you can help. You will be surprised by the responses you receive.

I can already hear some of you saying to yourself “you don’t know my boss, you don’t know this environment, you don’t know…” yup, you are right. I don’t know. I have no idea. Here’s what I do know, if you take the time to think through this post and send a list of thoughtful questions to someone whom you admire professionally, their response may surprise you. I also know that I’ve been blessed to be in some incredible organizations with some wicked tough challenges. Earlier in my career, these challenges would have torn me to bits – I simply wouldn’t have been ready for them. Heck, I’m still not ready for some of the challenges my clients face. However, I’m honest with myself about it, and I’m able to articulate the value that I can provide and just as important where I cannot provide value.

My hope for you, is that you will go through this process. And if you have gone through this type of process, why stop now? Go through it again and again. Then take an even more important step and go through this process in your personal life.

Speaking

This past weekend, I had the most challenging experience I’ve ever encountered as it relates to speaking in public.

I’ve been blessed with opportunities to speak at Dev Connections, SQL Saturday events, User Group events and of course, in my day to day life as a consultant I speak to differing audiences on a fairly regular basis.

I’ve read a lot over the years as it pertains to giving a good presentation, I’ve seen many good public speakers and some real duds as well.

However, I now have a new found appreciation for folks who speak to audiences on manners that aren’t technical or in general, targeted topics / concepts.

This past weekend I gave a eulogy.

The individual I gave it for was close to me, however, even if they hadn’t been, I believe that the same challenges would have existed.

It’s interesting – one of the first rules in public speaking is to “Know Thy Audience” – this is, of course critically important to being an effective speaker, consultant, employee, manager, CxO, etc…
So, what is the audience for a eulogy?

Well, it’s obvious, right?

Clearly, it’s full of people who knew the deceased – but then there are some wrinkles in the midst of that – differing relationships – from Mother, Daughter, Wife, Husband, Son, Father, Brother, Sister, 1st cousin, 8th cousin, grandkids, friends, colleagues, etc… and then in the midst of that wrinkle there are some very significant generational gaps. From toddlers to seniors who were born during the great depression.

Wow. Imagine preparing your favorite technical session with that audience in mind – and then, do it without a slide deck. Do it when EVERYONE is watching YOU. Do it when you see tears streaming down a cheek. Do it when there are tears streaming down your cheek and the only thing you can feel is a lump in your throat the size of a grapefruit.

The purpose of this post is to challenge you, the reader, to get up – be bold. Speak. It’s to challenge you in your personal life. Take that risk. I don’t know what it is, but take it. There is but this moment in life, seize it.
It’s tough, I’m not going to lie. However, it does get easier. You will get better. You’ve heard the old adage a thousand times “Practice makes perfect” – while I’m not sure about the “perfection” aspect of it, it certainly makes one better, a lot better.

Had I not faced my fear of speaking in front of my peers on technically related items, I’m not sure how the Eulogy would have turned out – and let’s face it, what’s more important in life, providing a good technical session or being able to prepare oneself for a life event that takes on a very real sense of importance?

I’m not trying to diminish or otherwise negate the importance of technical sessions – they have provided me with a great deal of knowledge over the years and I have certainly benefitted from them, however, having attended many of them and given more than a handful, it’s my experience that, having given a eulogy, my approach to giving technical sessions is going to change – perhaps drastically.

First and foremost, preparation.

In the normal course of giving a technical presentation, I will typically come up with what I think is a good idea. Then I’ll scribe an outline, put together the demo’s I believe illustrate the points / relevant technical details and then create some slides, add the detail to my outline and practice.

In the course of giving the eulogy – it was a very different experience. First, I ASKED many people what they thought of different ideas, I bounced around different stories, I THOUGHT a lot about the audience. I wrote some notes, scribbled down thoughts over the course of a day or two, then sat down and wrote it out.

Then I practiced. I practiced in a way I hadn’t before. I practiced speaking the eulogy out loud with my favorite music playing. I practiced it while driving the car. I practiced it walking around. I practiced it while hitting golf balls.
This was different for me – it had been my intent to practice in enough “distracted” environments that when it came time to do it “live”, I’d be ready for the grapefruit in my throat, I’d be ready for the faces in the audience who were looking at me.
And you know what? It worked. Was my delivery flawless? No. Did I recite the eulogy from memory without once checking my notes? No. Did I have some unexpected hiccups? Yup. Did they throw me? Nope.
Was I able to connect with every person in the Church? I haven’t a clue – but I do know that out of the ~ 175 folks in attendance, I was blessed to hear many of them say “thank you, it was so eloquent, etc…” Afterwards, I asked a couple of different generational groups what they heard me say – and guess what? They heard the same thing. Victory. In a big, huge way.

How many times have you walked out of a technical presentation with a different set of “facts” or even “ideas” about the subject matter than the guy who was sitting next to you? I can answer that for you, it happens very often.

Now, I’ll likely take some flak for this, and all I can say is “bring it on” – here you are: if your session has folks walking out with differing ideas of what you just spoke about, you weren’t as effective as you’d hoped. Yup, that’s right, as a speaker, you failed them, and you failed yourself.

I’m guilty of this. I’m absolutely certain that there’s been ambiguity in my sessions. I always chalked it up to “different people have different levels of understanding”, so, it’s not on me.

But guess what? It is.

Here’s the thing – when I give presentations to my clients, I can often anticipate the outcome of the conversation. If I cannot anticipate the questions which will come my way, based on information which I’ve delivered, then either I don’t know my subject area well enough or I haven’t done enough due diligence. If the “takeaways” from a client presentation are not 100 percent clear to the client and myself, I’ve failed us both and my time as being their consultant will surely come to an end at some point down the road.

This is where I see this fork in the road. Giving a presentation to a client or giving a eulogy both have different levels of meaning and importance. Giving a session at a conference doesn’t have the same types of pressure. There are different pressures with technical conferences – there’s the being grilled / vetted by your peers if you are wrong. There’s absolutely accountability from an accuracy of the presentation standpoint.

BUT,

There’s NOT any accountability from the perspective of WHAT you conveyed to your audience in terms of the HOW’s and WHEN’s and WHY’s.

This is really important and I’m *finally* getting to the heart of this post.

If you are going to give a presentation or if you are going to attend a presentation and the speaker makes a statement such as “this technology will save your DBA staff so much time” or “this new feature will help increase your performance” – stop. Full stop. Are you sure? How sure are you? I’ll answer this for you, you aren’t. Neither the speaker, nor the attendee.

However, as the speaker, you can be sure. As the attendee, it’s your responsibility to be sure.

It’s always amazed me how often I’ll give a presentation and the amount of questions at the end aren’t more numerous or specific with implementation details or their specific environment or challenge.

Know Thy Audience pertains to the speaker. Know Thy Self pertains to the attendee and Mr. / Ms. Attendee, if you don’t know the subject material being presented on then please do one of two things, either learn it before you attend the session or spend a significant amount of time after the session learning WHEN and WHY it *might* be a good idea to implement.

I cannot count, on both hands or feet, how many times I’ve seen terrific technology used in the wrong manner, resulting in botched software, poor quality, high TCO, headaches, nightmares, well, you get the idea.

This also comes back to an old post on technical debt I wrote once. It feeds into it perfectly and, for those of you who speak, you have a larger responsibility, and for those attendees, you also have a larger responsibility. Don’t feed the monster which is technical debt. While we’ve all done it, most often out of the best of intentions, there exists an opportunity to acknowledge the reality and learn from it.

So, here’s the challenge to the speakers out there – focus less on breadth and depth of a topic and focus more on the relevance of a topic. Attendees, do yourself and your respective companies a huge service and ensure that you are ready for the material which is about to be presented.

Index Aligned Partitions and Resolution

Most of us are aware of partitioning in SQL Server.  It was introduced with SQL Server 2005 and then further enhanced in later releases.  The value of partitioning a table can vary.  Some use it as a means of archiving off older data, others use it for the sake of performance gains, either way, there are some important things to keep in mind when one partitions a table.

One of those items brings us to the point of the post, aligning your non-clustered indexes on the partition scheme.  This is an important rule and more often than not it should be followed.

One of the reasons that it’s important is if you ever want to switch partitions, all of the indexes associated to that object must be aligned.  ref: http://msdn.microsoft.com/en-us/library/ms187526.aspx

Another reason is around maintenance.  If the indexes are partitioned then only the active partition (based on your partitioning strategy) needs to be maintained (re-org’d).  For larger indexes, it can become prohibitive to re-build these when they become fragmented.

Probably the best reason to have your indexes aligned is that the amount of reads to satisfy your queries will significantly drop when that index is used.

I recently came across a scenario where I found that there were a couple of tables where the indexes weren’t aligned with the partition scheme.  Instead of going through every partitioned table in the database, I wrote a quick query that would tell me which objects had indexes that weren’t created on the partition scheme for the table.

This query will also show you the usage of the indexes which can be interesting as well.  Without further ado, here’s the query:

SELECT Db_name(database_id) AS dbname, 
       o.name               object_name, 
       i.name               index_name, 
       i.type_desc, 
       ds.name, 
       ds.type_desc, 
       s.user_seeks, 
       s.user_scans, 
       s.user_lookups, 
       s.user_updates, 
       s.last_user_seek, 
       s.last_user_update 
FROM   sys.objects AS o 
       JOIN sys.indexes AS i 
         ON o.object_id = i.object_id 
       JOIN sys.data_spaces ds 
         ON ds.data_space_id = i.data_space_id 
       LEFT OUTER JOIN sys.dm_db_index_usage_stats AS s 
         ON i.object_id = s.object_id 
            AND i.index_id = s.index_id 
WHERE  o.TYPE = 'u' 
       AND i.Type in (1, 2) 
       AND o.name IN (SELECT a.name 
                      FROM   (SELECT ob.name, 
                                     ds.type_desc 
                              FROM   sys.objects ob 
                                     JOIN sys.indexes ind 
                                       ON ind.object_id = ob.object_id 
                                     JOIN sys.data_spaces ds 
                                       ON ds.data_space_id = ind.data_space_id 
                              GROUP  BY ob.name, 
                                        ds.type_desc) a 
                      GROUP  BY a.name 
                      HAVING COUNT(*) > 1) 
ORDER  BY object_name DESC  

That’s fun stuff… But then what happens when you actually want to FIX them?  If there are a lot of them, it makes for a fairly tedious task.  The code below will help.  Be absolutely certain that you check the options chosen.  For my database, I used MAXDOP = 32, ONLINE = ON, etc… Ensure that the the settings are appropriate for your environmnent… enjoy!

SET NOCOUNT ON

SELECT db_name(database_id) as DBName,o.name Object_Name,		
        i.name Index_name, 		
       i.Type_Desc as IDX_typeDesc ,DS.name , ds.type_desc
		,sch.name + '.'+o.name as SchemaName_TableName, CAST(NULL as varchar(128)) as partitonOrdinal
into #tmp1
 FROM sys.objects AS o		
      JOIN sys.indexes AS i		
  ON o.object_id = i.object_id 		
	JOIN sys.schemas sch on sch.schema_id = o.schema_id
	JOIN sys.data_spaces DS on DS.data_space_id = i.data_space_id		
	LEFT OUTER JOIN sys.dm_db_index_usage_stats AS s ON i.object_id = s.object_id AND i.index_id = s.index_id		

 WHERE  o.type = 'u'		
   AND i.type IN (1, 2) -- Clustered and Non-Clustered indexes
   AND O.NAME in 		
	(	
	SELECT a.name from 		
		(SELECT OB.NAME, ds.type_desc 
		 FROM sys.objects OB 
			JOIN sys.indexes ind on ind.object_id = ob.object_id 
			JOIN sys.data_spaces ds on ds.data_space_id = ind.data_space_id		
		GROUP BY ob.name, ds.type_desc ) a 
  group by a.name having COUNT (*) > 1
	)	
order by Object_Name desc;

update a
	set partitonOrdinal = c.name 
	FROM #tmp1 A JOIN 
	  sys.tables          t ON A.Object_Name = t.name  
	join sys.schemas s on s.schema_id = t.schema_id
	join  sys.indexes         i 
		  on(i.object_id = t.object_id 
		  and i.index_id < 2)
	join  sys.index_columns  ic 
		  on(ic.partition_ordinal > 0 
		  and ic.index_id = i.index_id and ic.object_id = t.object_id)
	join  sys.columns         c 
		  on(c.object_id = ic.object_id 
		  and c.column_id = ic.column_id)
	WHERE I.name = a.Index_name and a.IDX_typeDesc = 'clustered'    ;    

update a
	set name = b.name, partitonOrdinal = b.partitonOrdinal
	FROM #tmp1 a join #tmp1 b on a.object_name = b.object_name where b.idx_typedesc = 'clustered';



WITH index_cte (table_name, index_name, type_desc, index_columns, index_included_columns,
filter_definition, is_unique, is_primary_key, is_unique_constraint,fill_factor,
is_padded, allow_row_locks, allow_page_locks, ignore_dup_key, data_compression_desc,
no_recompute, partitionname, partitonOrdinal)
as(
select sch.name+'.'+obj.name [table_name], i.name [index_name], i.type_desc,
REPLACE(
(select '['+col.name+'] '+
	CASE WHEN k.is_descending_key = 1 THEN 'DESC'
	ELSE 'ASC'
	END as [data()]
from sys.index_columns as k
inner join sys.columns col
		on k.object_id=col.object_id and k.column_id=col.column_id
	where k.object_id = i.object_id
	and k.index_id = i.index_id	and k.index_id<>0 and k.is_included_column<>1
	order by key_ordinal, k.column_id
for xml path(''))
,'C [','C, [') as [index_columns],
isnull(
REPLACE(
	(select '['+col.name+']' as [data()]
from sys.index_columns as k
inner join sys.columns col
		on k.object_id=col.object_id and k.column_id=col.column_id
	where k.object_id = i.object_id	and k.index_id = i.index_id and k.index_id<>0 and k.is_included_column<>0
	order by key_ordinal, k.column_id

for xml path(''))
,'] [','], [')
,'') as [index_included_columns],

	ISNULL(i.filter_definition,'') as filter_definition,
	is_unique, is_primary_key, is_unique_constraint, fill_factor, is_padded, allow_row_locks, allow_page_locks, ignore_dup_key, 
	data_compression_desc, no_recompute, a.name, a.partitonOrdinal 
from sys.indexes as i
inner join sys.objects obj
	on i.object_id=obj.object_id
inner join sys.schemas sch
	on obj.schema_id=sch.schema_id
inner join sys.partitions part
	on i.object_id=part.object_id
	and i.index_id=part.index_id
inner join sys.stats stats
	on i.object_id=stats.object_id
	and i.index_id=stats.stats_id

JOIN #tmp1 a ON a.index_name = i.name AND a.object_name = obj.name and a.type_desc = 'rows_filegroup'

where i.name is not null and obj.type<>'S' and sch.name<>'sys' and i.index_id>1
)
SELECT
'CREATE ' +
CASE is_unique
	WHEN 1 THEN 'UNIQUE '
	ELSE ''
END
+ index_cte.type_desc +' INDEX ' + index_cte.index_name COLLATE DATABASE_DEFAULT +' ON ' +table_name +' ('+index_columns+')'+
CASE index_included_columns
	WHEN '' THEN ''
	ELSE ' INCLUDE (' + index_included_columns + ') '
END+
CASE filter_definition
	WHEN '' THEN ''
	ELSE ' WHERE ' + filter_definition
END+
' 
	WITH ('+
	'FILLFACTOR = ' + CASE fill_factor WHEN 0 THEN '100' ELSE CAST(fill_factor as varchar) END+
	', PAD_INDEX = ' + CASE is_padded WHEN 1 THEN 'ON' ELSE 'OFF' END+
	', ALLOW_PAGE_LOCKS = ' + CASE allow_page_locks WHEN 1 THEN 'ON' ELSE 'OFF' END+
	', ALLOW_ROW_LOCKS = ' + CASE allow_row_locks WHEN 1 THEN 'ON' ELSE 'OFF' END+
	', IGNORE_DUP_KEY = ' + CASE ignore_dup_key WHEN 1 THEN 'ON' ELSE 'OFF' END+
	', STATISTICS_NORECOMPUTE = ' + CASE no_recompute WHEN 1 THEN 'ON' ELSE 'OFF' END+
	', DATA_COMPRESSION = ' + data_compression_desc +
	', SORT_IN_TEMPDB = ON'+
	', ONLINE = ON'+
	', MAXDOP = 32'+
	', DROP_EXISTING = ON'+
	')' + '
	 ON  ' +  partitionname + ' (' + partitonOrdinal + ')
 GO'
as 'USE Database statement goes here (TNW)'
from index_cte
 order by table_name, index_cte.type_desc, index_cte.index_name

drop table #tmp1