IdeaBlade DevForce – Model Setup Walk-through – Step 2: The DevForce Projects

Our Problem:


We need to create an Enterprise application that is web-based, has an Excel-ish interface and pulls data from an Oracle database.

Technologies Chosen*:

  • Silverlight for the UI
  • Prism 2 for the modular framework
  • Unity for DI
  • IdeaBlade DevForce for the Business Object Layer and Async functionality
  • MS Entity Framework for the ORM
  • DevArt dotConnect for Oracle (EF Drivers)
  • Oracle 11 Database

* See here for our reasoning.

Prerequisites:

  1. Complete the procedures documented in Step 1
  2. Install IdeaBlade’s DevForce for Silverlight tool.
  3. Install the Silverlight development environment. This comprises:
    1. The Silverlight 3 Developer Runtime
    2. The Silverlight 3 Tools for VS 2008
    3. The Silverlight 3 SDK

Creating the DevForce model projects:


There will be two DevForce model projects, one will be a C# library project that runs on the server, the other will be a Silverlight library that runs in the Silverlight client. The DevForce framework takes care of wiring these two together with a couple of WCF services that are hosted in the Web application that hosts the DevForce components and the Silverlight app.

To create the projects:

  1. Open the Entity Framework model solution created in Step 1 of this walkthrough.
  2. Select Tools | DevForce Object Mapper
  3. DevForce naturally works with EDMX files (Model | Add Entity Model), but we have an EDML file from the devArt Entity Developer, so we have to use File | Open to get to it.

    Select File | Open, change the Files of type to All Files. Select the Model_Widget.edml file and click Open.
    At this point, DevForce has opened the EF model. Now, we have to tell it how we want the projects to be configured.

  4. First, click the New Project button for the Domain Model project.
    The Project Name needs to follow our assembly naming convention, so for this walkthrough, I am using Acme.BirdTrap.Model.Widget (with no suffix).
    I will leave the Project Type and Location at their defaults.
  5. Second, click the New Project button for the Silverlight project.
    The Project Name needs to follow our assembly naming convention, so for this walkthrough, I am using Acme.BirdTrap.Model.WidgetSL (with the SL = Silverlight suffix).
    I will leave the Project Type and Location at their defaults.
  6. Back on the new model tab:
    1. Set the Namespace to: Acme.BirdTrap.Model.Widget
    2. Set the EntityManager name to something more model specific, we use a standard of ModuleEntityManager as the application may need to work with multiple modules’ EntityManagers at the same time.
      I have selected to name the Entity Manager: WidgetEntityManager.

    New Devforce Project Settings

    [Note: See Ward Bell’s essay on the topic of Large Models for more about splitting your model and using multiple Entity Managers.]

  7. Select the Model_WidgetEF.edml branch of the Model tree on the left. You will see the settings available for this EF Model.
    1. Change the DataSourceKeyName to something more model specific, we use a standard of ModuleDataSource as each module/model needs a different DataSource definition.
      I have selected to name the Data Source Key: WidgetDataSource
    2. This is also the place where you inject your own base Entity class implementations into the DevForce model. At the very least, you should create a BaseEntity class where you can put common Entity functionality. Click the Injected Base Types button and add Acme.BirdTrap.Utility.Persistence.BaseEntity to the list, setting it as the default. (In a future posting, I will be examining the Injected Base Types in greater depth.)
    3. The name pluralizer is a great feature of DevForce but it is completely irrelevant to us as devArt’s Entity Developer tool has taken care of pluralization for us.

  8. Click the Save button on the toolbar. DevForce will now create and initialize the two projects we have defined. When it has finished, close the DevForce Object Mapper.
  9. Next, we need to create the BaseEntity class we added to the Injected Base Types earlier.
    1. In the new C# project, Acme.birdTrap.Model.Widget, add a new class named Acme.BirdTrap.Utility.Persistence.BaseEntity that inherits from IdeaBlade.EntityModel.Entity
      using IdeaBlade.EntityModel;
      
      namespace Acme.BirdTrap.Utility.Persistence
      {
          public class BaseEntity : Entity
          {   
          }
      }
    2. In the new Silverlight project, Acme.birdTrap.Model.WidgetSL, add an Existing Item, then navigate to the BaseEntity class you just created in the C# project, drop down the Add button and choose Add as Link.
  10. I bet you wish we were done at this point. We are not. DevForce made some assumptions in generating the projects that we need to fix.

    First, and really only a cosmetic issue, DevForce created a Solution Folder for the DomainModel. You can choose to keep this or not. This whole solution is for my DomainModel, so I do not need this new folder. If I incorporate the Model projects into a larger Module level Solution, then I will use a Solution Folder for the Model projects in that solution, but I do not need one here. To remove the folder, I drag the two new projects up out of the folder into the Solution level and then Remove the folder.

  11. Second, and by far more serious, DevForce pulled in a bunch of referenced components, some of theirs, some of the .Net Framework’s and some from the Silverlight SDK.
    There are three places where referenced assemblies can safely live:

    1. \WINDOWS\Microsoft.NET\Framework\**.*
    2. \Program Files\Reference Assemblies\**.*
    3. Your solution’s dependencies\**.*

    1. Examining the C# project (Acme.BirdTrap.Model.Widget), we find the IdeaBlade assemblies are referenced from Program Files. We can change that by: first, removing the Ideablade.Core, Ideablade.EntityModel, Ideablade.Validation,assemblies; then adding them back from the dependencies\IdeaBlade DevForce\ folder.
    2. Examining the Silverlight project (Acme.BirdTrap.Model.WidgetSL), we find the IdeaBlade Silverlight assemblies are referenced wrong, which can be corrected as above, only adding back from the dependencies\IdeaBlade DevForce\SilverlightAssemblies folder this time.
      However, we also find that the System.ComponentModel.DataAnnotations assembly is being pulled from \Program Files\Microsoft SDKs\ folder. Remove it and add it back from dependencies\Silverlight\Libraries\Client\.
  12. That’s it, right? We’ve created the files and all is well. If you try to Build your solution right now, you will get errors originating from the EF project. But it was fine before we started, and we didn’t change it — or did we?
    In fact, we did. When you point the DevForce Object Mapper at the EF project’s EDML file, DevForce adds a bunch of attributes to some of the elements in that file. In doing so, the DevArt Entity Developer notices the EDML has changed and wipes the generated *.designer.cs file. To restore this, you need to re-open the EDML file and regenerate the generated code. But here’s the rub, if you open the EDML file from within Visual Studio, the devArt Entity Developer will only regenerate the code if you make changes to the model. But we don’t want to make changes to the model.
    Instead, open the EDML file from Windows Explorer. This will open the Entity Developer and reveal a Generate Code button on the toolbar:
    Generate Code button
    Clicking this button, will regenerate the missing generated code without requiring a model change. You can then exit the Entity Developer.
  13. DONE.

Conclusion


In this part of the walk through, we have created the pair of DevForce C# and Silverlight projects that contain our Domain Model.
The solution successfully Builds and the EF unit test we created in Step 1 still passes. Now we need to create a pair of Unit Test projects to ensure that the DevForce framework is configured properly and can successfully CRUD the database through the EF layer. But that will have to wait until the next part of the walk through.

Until then: I’m off to take the dog for a walk! Woof!

Moving Code from the Developer’s Integration Environment to the QA Environment

When we’ve completed coding (and, if we’re feeling especially adventurous, testing said code), we promote our locally developed code to the Dev server. This serves as our integration environment. CruiseControl.Net watches our SCM for any changes promoted to the DEV branch, then it:

  1. Updates the configuration files to point to the Dev server instead of the developer’s local workstation;
  2. Builds our solution in Debug mode (for more verbose logging during the unit test runs);
  3. Runs the unit tests;
  4. Builds the solution again in Release mode;
  5. Runs the unit tests again to ensure they work in Release mode too; and
  6. Finally, publishes the necessary files to IIS.

Today, I need to figure out how much has to change to get this project onto our QA server.

The Problem

Pointing the QA environment to the QA database.

The biggest difference between the process described above and the process to move to QA is going to be changing the back end database that the solution points to. I.e. The Developer’s local workstation and the Dev server both point to the Dev database. (It’s an Oracle back end, so we do not have a separate local and server database — even if we wanted to, but I’m not sure we would.)

ORACLISM

An aside about Oracle schema ownership. As a SQL Server developer, there are several Oraclisms that confound me. I’m not sure that this is one of them, but here’s my take on it: In SQL Server, most databases are created by dbo and most tables are created as dbo. Security is then applied to users and groups to grant access to various tables. If you need to create a Dev database and a QA database for the Production database named AccountingSystemDB, you might create a new database, again as dbo, called AccountingSystemDEV and another called AccountingSystemQA. In Oracle, this notion of separate databases is superseded by a notion of separate schemas WITHIN a single database. There is no obvious corollary to dbo. The Oracle system user serves a very different purpose, perhaps relating more to the notion of sa in SQL Server. Instead, an Oracle DBA will create a Schema Owner (i.e. a user?) and then create the Schema as that owner, granting rights to that Schema to users.

For example, the Oracle Server has a single Database instance, within which, you will see three schemas named ACC_SYS_DV, ACC_SYS_QA and ACC_SYS_P. Within each schema, you will find that schema’s version of the tables. I.e. ACC_SYS_DV.ACCOUNT, ACC_SYS_QA.ACCOUNT and ACC_SYS_P.ACCOUNT. Also, a lot of database apps, use a single Application User Id to access the database, relying on the Application to implement security. (Or at least, ours does.) For each of the three environments, this Application User logs in to one of the three Schemas.

I am concerned that Entity Framework knows the Database and Schema in the connectionstring in the app.config for the ModelEF project:

 ...
 <edmKeys>
  <edmKey
   name="DataSource"
   connection="metadata=res://ModelEF/ModelEF.csdl| res://ModelEF/ModelEF.ssdl| res://ModelEF/ModelEF.msl; provider=Devart.Data.Oracle; provider connection string=" User Id=APPDV; Password=*****; Server=devTns.world; Home=oraodac11g_home1;Persist Security Info=True""
   containerName="ModelEF.Entities">
   ...
  </edmKey>
  ...
 </edmKeys>
</ideablade.configuration>

Above we see the Oracle server instance devTns.world, and the Dev database schema owner APPDV. There is also a Home= entry pointing to the developer’s OraHome that needs to be removed.

EF then maps the Schema owner into the EDML (EDMX) file:

...<EntitySet
  Name="ACCOUNTs"
  EntityType="ModelEF.Store.ACCOUNT"
  store:Type="Tables"
  Schema="APPDV" <=NOTICE DV SCHEMA OWNER IS HERE
  Table="ACCOUNT" />...

If this was a SQL Server solution, the schema owner would be the same (whether it is dbo or not) in the AccountingSystemDEV and AccountingSystemQA databases. In Oracle, this is not the case, so I am curious if I will need to update the Schema tag in the EDML (EDMX) . If so, the promotion to QA will force a rebuild, not just a reconfigure.

The Ideablade configuration also knows the Database and Schema in the connection attribute of the edmKey tag in the Model project’s app.config and in the web host project’s web.config:

<configuration>
 <connectionStrings>
  <add name="Model.EFConnectionString"
   connectionString="metadata=res://*/ModelEF.csdl| res://*/ModelEF.ssdl| res://*/ModelEF.msl; provider=Devart.Data.Oracle; provider connection string=&quot; User Id=APPDV; Password=*****; Server=devTns.world; Home=oraodac11g_home1; Persist Security Info=True&quot;"
   providerName="System.Data.EntityClient" />
 </connectionStrings>
</configuration>

[It turns out, I do need to change the tags, change the connection strings, and regenerate the pregenerated views.]

A SOLUTION

Since we are using CruiseControl.Net (CCNet) for our automated build, and since we are using Nant for some of the less “build-ish” and more “configure-ish” tasks, I was able to add a Nant task in the CCNet project configuration that does a string replace in all the EDML (EDMX) and .config files, and then uses the T4 Template technique to recreate the Pregenerated Views.

In order to use the T4 Template attached to the ADO .Net blog, I had to make two changes because we are using Devart Entity Developer. First, the .tt file needs to be renamed from ModelEF.Views.tt to ModelEF.PregeneratedViews.tt. Second, the following line in the .tt file has to be changed:

string edmxFileName = Path.GetFileNameWithoutExtension(this.Host.TemplateFile).ToLowerInvariant().Replace(".views", "") + ".edmx";

Should be replaced with:

string edmxFileName = Path.GetFileNameWithoutExtension(this.Host.TemplateFile).ToLowerInvariant().Replace(".pregeneratedviews", "") + ".edml";

Here is the CCnet task in the CruiseControl.Net ccnet.config file to execute the Nant task:

<project name="PromoteToQA">
 ...
 <tasks>
  <nant>
   <buildFile>ChangesBeforeBuild-QA.build</buildFile>
   <executable>E:\Program Files\nant-0.86-beta1\bin\nant.exe</executable>
  </nant>
 </tasks>
 ...
</project>

And here is the Nant task itself:

<project name="ChangesBeforeBuild-QA" default="run">

 <target name="run">
  <call target="change Database" />
 </target> 

 <target name="change Database">
  <foreach item="File" property="filename">
   <in>
    <items>
     <include name="**/*.edml" />
     <include name="**/app.config" />
     <include name="**/web.config" />
     <include name="**/*EF.Designer.cs" />
     <include name="**/*EF.edps" />
    </items>
   </in>
   <do>
    <attrib file="${filename}" normal="true" />
    <loadfile file="${filename}"
     property="file.textContent">
     <filterchain>
      <replacestring from="APPDV" to="APPQA" ignorecase="true"/>
      <replacestring from="*****" to="*****" />
      <replacestring from="devTns.world" to="qaTns.world" ignorecase="true"/>
      <replacestring from="Home=oraodac11g_home1;" to="" />
     </filterchain>
    </loadfile>
    <echo message="Writing Database Change to: ${filename}" />
    <echo file="${filename}">${file.textContent}</echo>
   </do>
  </foreach>
 </target>
 
 <target name="Recreate Precompiled Views Files">
  <!-- unlock the view files -->
  <foreach item="File" property="filename">
   <in>
    <items>
     <include name="**/*.PregeneratedViews.cs" />
    </items>
   </in>
   <do>
    <attrib file="${filename}" normal="true" />
   </do>
  </foreach>
  <!-- transform the *.PregeneratedViews.tt files to recreate the *.PregeneratedViews.cs files -->
  <foreach item="File" property="filename">
   <in>
    <items>
     <include name="**/*.PregeneratedViews.tt" />
    </items>
   </in>
   <do>
    <echo message="Transforming: ${filename}" />
    <exec program="dependencies\TextTransform\TextTransform.exe">
     <arg value="${filename}"/>
    </exec>
   </do>
  </foreach>
 </target>
 
</project>

And that’s it. Before CruiseControl.net starts running the actual tasks, it runs this Nant task to prepare the source files for the QA environment.

Why does he keep saying EDML instead of EDMX?

In the Microsoft Entity Framework (EF), the GUI for the Entity Model is called the Entity Designer.  When you create an Entity Model, the model itself is stored in an XML file named model.EDMX.

The EDMX file is: 

a) an XML representation of the Entities and Relationships in your Database (the Storage Model, or SSDL), your application (the Conceptual Model, or CSDL), and the mapping from one to the other (the  Mapping Specification, or MSL); and

b) a repository for the metadata that defines the Entity Designer Diagram layout.

And this works out just fine for SQL Server.  Microsoft provides a nice little driver that is EF compatible, they provide a nice little GUI to create your Model with, and SQL Server tables — if named “normally”, i.e. Customer, Account, etc. — map beautifully to your object model.

This is NOT the case for Oracle.  Oracle still holds its nose whenever anyone says “Microsoft”, so they haven’t, and won’t, provide a driver for EF.  Heaven forbid they should make it easy to use Visual Studio and the .Net framework to work with their data.  Nope — Java, Eclipse, and the Oracle Application Framework are their preferred toolset.

So what is a .Net developer to do?  How can one create an EF model from an Oracle database?

ANSWER:  DevArt

DevArt has gone to the trouble of creating ADO.Net drivers for a variety of databases, including SQL Server and, da-da-da-da, Oracle.

I have been using DevArt’s dotConnect for Oracle drivers for EF purposes for about a year now.  In the early days, they were quite quirky.  I posted to their forums regularly.  Every single problem, suggestion, requirement or wish list item that I posted has been implemented, usually in their next release.  (They post quiet releases every 4 to 6 weeks.  Yay for Agile Development.)  Their latest version (5.35.79 as of this writing) works flawlessly for my purposes.

Anyway, included with their dotConnect drivers, is the DevArt Entity Developer.  This tool ROCKS!  Over the past year, it has matured to the point that I have completely abandoned using the MS Entity Designer for anything.  Instead, I create  and maintain my Entity Model’s using the DevArt Entity Developer. 

Anyway, in order to not step on anyone’s toes, DevArt saves the Entity Model as an EDML file, not an EDMX file.  (Yes, this is the “exact same” Entity Model XML that MS saves.)  They also store a bunch of metadata and project selections in a set of generated files behind the EDML file.  And, as of December, you can select to have the Entity Developer automatically generate the precompiled views file too.  (MS doesn’t do that and provides a 30-step process to accomplish it — A problem that has spawned many web posts.)

One of my favorite features of the DevArt Entity Developer over the MS one, has to do with an Oraclism that just bugs me. 

ORACLISM
Oracle DBA’s, (by tradition, by culture, or by requirement, I’m not sure which), often name database objects in all caps with underscore delimiters.  For example, a table named CustomerAccountType in SQL Server, will probably be named CUST_ACCT_TYP in Oracle.  (There’s an unreasonably low limit to how long the name can be in Oracle, so the DBAs often have to truncate or compress names to make them fit.) 

So, if you have a table defined thus:

CREATE TABLE  CUST_ACCT_TYP (
CUST_ACCT_TYP_ID NUMBER(10) NOT NULL,
TYPE_DESCRIPTION VARCHAR(20) NOT NULL)

Now, you drag and drop that table into the MS Entity Designer*, you will get an object defined thus:

Class:     CUST_ACCT_TYP
Property:  CUST_ACCT_TYP_ID, Double
Property:  TYPE_DESCRIPTION, String

Not a very “.Net-ish” way of defining a class, is it?  Also notice the NUMBER(10), an integral number type, is mapped to a Double.

DevArt’s Entity Developer, however, does a grand job of converting all those names, so you end up with an object defined thus:

Class:     CustAcctTyp
Property:  CustAcctTypId, Int64
Property:  TypeDescription, String

Notice how it even got the data type right for the Id. 

Now, if only I could UPDATE my model (instead of recreating the affected parts of it) when I change the Tables and Views in Oracle… 🙂

* By the way, you still need the DevArt drivers to do  this.