Tracking Technical Debt in ReSharper - Follow-up

Wow!  The good folks at JetBrains found me and gave me a link!  Thanks guys!

If you remember, we're still in the land of waterfalls, where everything has a signature and an arbitrary deadline, and a long schedule littered with milestones instead of actual released software.  While marching toward that deadline, we've built up a lot of techinical debt, and we've been tracking it in ReSharper for about a month now.  Well the time has finally come when we think we can begin to pay off some of that debt.  We decided that since we had finished all, or at least most, of the features we set out to build that we could afford to spend some time focusing on those items we could have implemented better during development. 

Our tracking scheme paid off quite nicely.  There was one refinement that helped some, a Live Template for a //DEBT tag, like so:

//DEBT - $type$.$member$ - $END$

The $type$ variable is set to the "Containing type name" macro, and the $member$ variable is set to the "Containing type member name" macro.  Did the same thing for the //HACK and //TODO tags.  We used the TODO Explorer to dump our //DEBT out to a list we could email around, and no matter how the list was sorted the type and member info made it easy to see where our //DEBT was. 

I have to say, once we introduced the idea of tracking //DEBT right there in our code, it became really easy to convince other developers on our team to focus on completing features rather than get bogged down in endless optimization.  A little switch flipped in everyone's head once we realized that we could rely on our //DEBT list to remind us of things we need to come back to again.

 

Share this post :

Parsing YAML files in Ruby - Part 2

I wish I had a clip reel I could roll after a dude with a really deep voice said, "Previously on Parsing YAML files in Ruby".  But I don't.  So here's a link.

Ruby is Narnia.  I spend my real life in c#, a perfectly serviceable language.  I'm comfortable there, I kind of know my way around, and I've come to depend on it to make my living.  But when I have a few spare moments here and there, I get to wander off into this magical fairy-land and have adventures with strange and wonderful creatures.  Like the YAML.  In case you forgot what our YAML looks like, here he is:

   1: ---

   2: shared paths:

   3:   build share : \\buildshare.mydomain.com\Builds

   4:  

   5: local paths:

   6:   references  : \references

   7:  

   8: custom assemblies:

   9:   - location: \Dev\Components\Business\Core\Trunk\Latest\Debug

  10:     assemblies:

  11:       - name : MyNamespace.Core

  12:         files:

  13:           - binary    : MyNamespace.Core.dll

  14:           - debug     : MyNamespace.Core.pdb

  15:           - document  : MyNamespace.Core.xml

  16:       

  17:   - location: \Dev\Components\Framework\Trunk\Debug

  18:     assemblies:

  19:       - name : MyNamespace.Framework.Core

  20:         files:

  21:           - binary    : MyNamespace.Framework.Core.dll

  22:           - debug     : MyNamespace.Framework.Core.pdb

  23:           - document  : MyNamespace.Framework.Core.xml

  24:  

  25: vendor assemblies:

  26:   - location: \vendor\DotNet Commons\Logging\2.0

  27:     assemblies:

  28:       - name : Dotnet.Commons.Logging

  29:         files:

  30:           - binary    : Dotnet.Commons.Logging.dll

  31:  

  32: testing assemblies:

  33:   - location: \vendor\Nunit\2.4.3

  34:     assemblies:

  35:       - name : NUnit.Framework

  36:         files:

  37:           - binary    : nunit.framework.dll

  38:         

  39:   - location: \vendor\Rhino.Mocks\3.3.0.906

  40:     assemblies:

  41:       - name : Rhino.Mocks

  42:         files:

  43:           - binary    : Rhino.Mocks.dll

  44:           - document  : Rhino.Mocks.xml

  45: ...

 

Last time I told you how easy it was to access data in *.yml files in Ruby.  I've taken idea that a little further, and I cooked up this class:

   1: require 'yaml'

   2:  

   3: class References

   4:   attr_accessor :debug_mode

   5:   def initialize(references_file_name='references.yml',debug_mode=true)

   6:     @refs = open(references_file_name) {|f| YAML.load(f) }

   7:     @debug_mode = debug_mode

   8:   end

   9:   def shared_root_directory

  10:     @refs['shared paths']['build share']

  11:   end

  12:   def get_filenames(assembly_list_name, *file_types)

  13:     get_node(@refs, assembly_list_name) do |assembly_list|

  14:       assembly_list.each do |packing_list|

  15:         get_node(packing_list, 'assemblies'){|assembly| get_names(assembly, packing_list['location'],file_types){|filename| yield filename}}

  16:       end

  17:     end

  18:   end

  19:   private

  20:   def concatenate(*locators)

  21:     concatenated = String.new

  22:     locators.each { |locator| concatenated << (locator =~ /\A(?!\\)/ ? '\\' : '') << locator.sub(/\\\Z/, '') }

  23:     return concatenated

  24:   end

  25:   def get_names(assembly, path,file_types)

  26:     get_node(assembly, 'files') do |file| 

  27:       parse_filenames(file,file_types){|filename| yield concatenate(shared_root_directory,path,filename)}

  28:     end

  29:   end

  30:   def get_node(data_store, find_key)

  31:     yield data_store[find_key] if data_store.kind_of? Hash

  32:     data_store.each{|node| get_node(node, find_key){|subnode| yield subnode}} if data_store.kind_of? Array

  33:   end

  34:   def parse_filenames(file_node,file_types)

  35:     file_node.keys.each {|key| yield file_node[key] unless filter(key,file_types)} if file_node.kind_of? Hash

  36:     file_node.each{|element| parse_filenames(element,file_types){|value| yield value}} if file_node.kind_of? Array

  37:   end

  38:   def filter(key,file_types)

  39:     (!@debug_mode && key=="debug") || (!file_types.include?(key) unless file_types.empty?)

  40:   end

  41: end

With this References class, you can do something like this (pay attention - here's where it starts to get cool):

refs = References.new
refs.get_filenames("custom assemblies","binary"){|filename| puts filename}

And you get something like this:

\\buildshare.mydomain.com\Builds\Dev\Components\Business\Core\Trunk\Latest\Debug\MyNamespace.Core.dll
\\buildshare.mydomain.com\Builds\Dev\Components\Framework\Trunk\Debug\MyNamespace.Framework.Core.dll

Let's start with get_node() on line 30.  This method is an iterator.  I don't know why, but it took a long time for the lightbulb to go off in my head over Ruby's usage of the yield keyword.  Turns out, it works just like all the Ruby books say it does.  Really, why would they lie?  In this case, on line 31, we're getting the value located in an element in the data_store hash picked out by the find_key variable, and yielding that value back to the calling method.  And that calling method better have a code block to execute once it receives a value, or we're gonna get a big ol' runtime exception.  For the get_filenames() call in our script, on line 14 we're saying, "Look in the top-most hash in the references.yml file and find me a node with a key called 'custom assemblies'".  Remember: the way our YAML file is laid out, it's just a big, weird hash of arrays and hashes.  We have to write code to ferret out the info we want, and in this case ultimately we want a list of filenames.

There's another interesting thing happening on line 31.  There's an if statement at the end of the line.  If you tried to get away with something like that in c# land, they'd lock you up and throw away the key.  Ahh, but here in Narnia, animals talk, trees walk, and all sorts of silly things happen.  You can even say "hey, do this thing if this other thing is true", they way people do.  No fussy brackets, or parentheses, or overly strict formatting rules to worry about.  Line 31 takes care of the case when data_store is a hash.  If data_store is not a hash, it's an array of hashes, and we take care of that on line 32, using a little recursion magic to get at the hash in each element of the array.

I think of the contents of 'custom assemblies' as an assembly_list, and each assembly_list contains packing_lists, each with a location and a list of assemblies at that location.  Each assembly can have more than one file associated with it - in this case I've listed the binary dll, the debug symbol pdb file, and the xml document associated with our custom assemblies.  I'm asking References to get just the binary files in 'custom assemblies'.  Now that I've got the 'custom assemblies' node, I already know that the assembly_list is an array, so I can just iterate through it with .each to get each packing_list. (That's the reason I wrote get_node() in the first place - as I was learning about YAML and Ruby, I wasn't sure what object types I was dealing with as I drilled down through the YAML file.  I could probably simplify the get_node() iterator now that I understand the structure of the file better, but I'll save refactoring for another time.)   A packing_list is a hash containing a 'location' that can have multiple 'assemblies'.  Line 15 says, "Get me the filenames for every assembly in this packing_list, and I'm really only want the ones that match this list of file_types".

The * in front of file_types in the get_filenames() signature makes it an optional parameter.  You don't have to specify the type of file you're looking for, and if you don't then you get back everything.  But if you do specify a list of file_types, that list is used in the filter() method, which is called on line 35.  Another cool Ruby-ish way of saying something: using the unless keyword.  Line 35 says, "Take a look at all the keys in the file_node hash, and give me back the file_node value for each key unless the key should be filtered out."

I'm hoping to put plain old Ruby classes and YAML together with Rake so that I can sweep angle brackets out of my life forever, and I'll post my progress as I learn.  Now I'm sure that there are better ways of expressing these things in Ruby.  But I'm new here.  I'm still enjoying my Turkish Delight and hot tea.  I still have a lot to learn about Narnia, but for now it's back to the real world. 

Share this post :

Introducing Developer Testing Into an Existing Team - Project Manager Thoughts

Last time, I posted our developer's thoughts on the answer to this question: What was the tipping point for you that helped you decide automated developer testing was a worthwhile thing?

Here's what one of our project managers thought:

dalesmithtx: I've asked some of our developers that question, but I'm curious about your perspective as a project mangler, err..., I mean manager

 Project Manager: The past weeks on [our current project] have shown the value

 Project Manager: for sure

 Project Manager: The challenge is getting the business folks to agree on that value

 dalesmithtx: What value have you been able to see?

 Project Manager: We saw that we could refactor and not miss a beat

 Project Manager: assumes that your project is going to have time to refactor. When you only get one shot at integration testing that's a challenge

 Project Manager: too often we follow the path of (1) code in silos (2) come together for a few days of integration (3) jump into QA

 dalesmithtx: right

 Project Manager: in that case the value goes way down because any refactoring the team does is in their spare time as they fix bugs and prepare for launch

 dalesmithtx: Do you think you are going to be able to sell it to the business folks?  Or do you see a need to do so?

 dalesmithtx: (from my perspective, as long as they're getting sausage, why do they care what happens in the sausage factory?)

 Project Manager: my last point was it still has value on the NEXT project or even the next hotfix. It increases your chances for quality follow on releases

 dalesmithtx: sure

 Project Manager: understood but if they KNOW about it they are going to fight to give you less time as you're doing work that could be cut

 dalesmithtx: I dig what you're saying

 Project Manager: If you just build it in and don't draw attention to it ... then yes

 Project Manager: build it into the schedule I mean

 dalesmithtx: At some point when we have an hour or so, I would really like for the [our current project] and [our other current project] teams to sit down and talk through a "lessons learned" session on this stuff

 dalesmithtx: Not so much a post-mortem specific to our projects,...

 Project Manager: right

 Project Manager: a technical seminar sorta

 dalesmithtx: But a post-mortem about the development techniques we used in an agile setting and in a waterfall setting

 dalesmithtx: pros and cons

 dalesmithtx: right

 Project Manager: yep I mean it had a cost in [our current project] but since the work so far has been so lop-sided on the UI side compared to the rest of the engineering folks, we had time

 dalesmithtx: right - that's an important point...

 dalesmithtx: I want to claim victory, but I want to claim it on solid ground, not by hiding behind latency elsewhere in the project

 dalesmithtx: I guess I'm looking for a real measure of the value that developer testing has brought to us so far, and what we think we need to change if we expect to get more value out of it on future development

 Project Manager: ok

 Project Manager: yeah I agree I don't like hiding tasks

Share this post :