Abstract Data Sources

Morfik Abstract Data Sources give you absolute freedom to bring information in from any source into your application or website. Abstract Data Sources provide a mechanism through which the developer by writing a couple of event handlers on the server side of the application can make any information stream appear as a table in the application. This allows all the functionality that is available for visually creating rich application interfaces based on the data sources to work with just about any data source.

The first step in defining an Abstract Data Source is to create a Table of custom type. Standard table objects represent actual database tables in the default database or in an external database. Custom tables on the other hand define the structure of data to be handled by the Morfik Framework as if it where a table in all places where data binding is possible, but which actually relies in user supplied code to obtain and update data, through a pair of event handler procedures.

To create an Abstract Data Source, go to the Power menu and choose the option to create a New Item|New Table. The small dialog box that will be displayed when you select this option will allow you to choose between creating a Standard or a Custom table. You should select the Custom option.

New Table Dialog

Morfik will display a grid where you can enter the names and types of the fields that you will wish your data source to have. It is important to have in mind that this data source will have no data storage of its own.

Abstract Table Designer

Once an Abstract Data Source is defined and the supporting code implemented, the Morfik Framework and the Morfik development environment will treat it basically in the same way as a regular data source. This means that you will be able to design your application in the same way you would, if it were accessing data from a table in the built-in database.

Using Abstract Data Sources you can write server side code that recovers data from any data source and presents it as a regular database table. In order to retrieve information from an external source and make it available as a data source you must implement the OnLoadDataevent on the server side and "insert" the data into a record set which will then be treated as a regular table by the Morfik Framework, on the browser side of your application or website.

In order to demonstrate how this can be done and just how powerful this feature is, a sample that displays pictures from Flickr as if they were on a regular table in the project's default database.

A couple of events on the server side code are responsible for handling the data input and output of an Abstract Data Source. In this below is included a code snippet with an implementation of the OnLoadData event for an abstract data source. This example retrieves publicly available pictures from Flickr and returns them as if they were stored in a local table.

The OnLoadData Event - This event is called when the RecordSet is opened. The developer can use this event to load the data source with information from a variety of sources such as: Web Service results, File system directory listing, contents of a text or XML file, etc.

Morfik Pascal code for requesting images from Flickr and returning them as content of a data source

FX Code

Procedure Table1Provider.WebTableLoadData(Recordset: TAnyDBRecordset);
Var
    Sl       : TStrings;
    Title    : String;
    SmallUrl : String;
    LargeUrl : String;
    i        : Integer;
 
    Function ExtractValue(Const S, Subst : String) : String;
    Var
        P : Integer;
    Begin
        P := Pos(Subst + '=', S);
        If P <> 0 Then
        Begin
            Result := Trim(Copy(S, P + Length(Subst) + 1, MaxInt));
            If Result.StartsWith('\"') Then
                Delete(Result, 1, 2);
            P := Pos('\"', Result);
            If P <> 0 Then
                Result := Copy(Result, 1, P - 1);
        End;
    End;
 
    Function ReplaceSuffix(Const S : String) : String;
    Begin
        Result := ChangeFileExt(S, '');
        Result[Length(Result)] := 's';
        Result := Result + ExtractFileExt(s);
    End;
 
    Function RemoveSuffix(Const S : String) : String;
    Begin
        Result := ChangeFileExt(S, '');
        Delete(Result, Length(Result) - 1, 2);
        Result := Result + ExtractFileExt(s);
    End;
 
    Function ExtractPicture(Const S : String; Var Title, SmallUrl, LargeUrl : String) : Boolean;
    Var
        Url : String;
    Begin
        Title := ExtractValue(S, 'title');
        Url := ExtractValue(S, 'img src');
        Result := Url <> '';
        If Result Then
        Begin
            LargeUrl := Url;
            SmallUrl := Url;
            Url := ChangeFileExt(Url, '');
            If Url.EndsWith('_m') Then
            Begin
                SmallUrl := ReplaceSuffix(SmallUrl);
                LargeUrl := RemoveSuffix(LargeUrl);
            End;
        End;
    End;
 
Begin
    Sl := TStringList.Create;
    Try
        If HttpGetText('http://api.flickr.com/services/feeds/photos_public.gne?en-us&format=csv', Sl)
           And (Sl.Count > 0) Then
        Begin
            For i := 0 To Sl.Count - 1 Do
            Begin
                If ExtractPicture(Sl[i], Title, SmallURL, LargeURL) Then
                Begin
                    Recordset.Append;
                    Recordset.FieldByName('Title').AsString := Title;
                    Recordset.FieldByName('ThumbUrl').AsString := SmallUrl;
                    Recordset.FieldByName('PictureUrl').AsString := LargeUrl;
                    Recordset.Post;
                End;
            End;
        End;
    Finally
        Sl.Free;
    End;
End;

        Public Message Sub WebTableLoadData(Recordset As TAnyDBRecordset)
            Dim Sl As TStrings
            Dim Title As String
            Dim SmallUrl As String
            Dim LargeUrl As String
            Dim i As Integer
 
            Private Function ExtractValue(Const S As String, Const Subst As String) As String
                Dim P As Integer
                P = Pos(Subst & "=", S)
                If P <> 0 Then
                    Result = Trim(Copy(S, P + Length(Subst) + 1, MaxInt))
                    If Result.StartsWith("\""") Then Delete(Result, 1, 2)
                    P = Pos("\""", Result)
                    If P <> 0 Then Result = Copy(Result, 1, P - 1)
                End If
            End Function
 
            Private Function ReplaceSuffix(Const S As String) As String
                Result = ChangeFileExt(S, "")
                Result(Length(Result)) = "s"
                Result = Result & ExtractFileExt(S)
            End Function
 
            Private Function RemoveSuffix(Const S As String) As String
                Result = ChangeFileExt(S, "")
                Delete(Result, Length(Result) - 1, 2)
                Result = Result & ExtractFileExt(S)
            End Function
 
            Private Function ExtractPicture(Const S As String, ByRef Title As String, 
                                            ByRef SmallUrl As String, ByRef LargeUrl As String) 
                                            As Boolean
                Dim Url As String
                Title = ExtractValue(S, "title")
                Url = ExtractValue(S, "img src")
                Result = Url <> ""
                If Result Then
                    LargeUrl = Url
                    SmallUrl = Url
                    Url = ChangeFileExt(Url, "")
                    If Url.EndsWith("_m") Then
                        SmallUrl = ReplaceSuffix(SmallUrl)
                        LargeUrl = RemoveSuffix(LargeUrl)
                    End If
                End If
            End Function 
 
            Sl = New TStringList()
            Try
                If HttpGetText("http://api.flickr.com/services/feeds/photos_public.gne?en-us&format=csv", Sl) 
                   and (Sl.Count > 0) Then
                    For i = 0 To Sl.Count - 1
                        If ExtractPicture(Sl(i), Title, SmallUrl, LargeUrl) Then
                            Recordset.Append()
                            Recordset.FieldByName("Title").AsString = Title
                            Recordset.FieldByName("ThumbUrl").AsString = SmallUrl
                            Recordset.FieldByName("PictureUrl").AsString = LargeUrl
                            Recordset.Post()
                        End If
                    Next i
                End If
            Finally
                Sl.Free()
            End Try
        End Sub

        public message void WebTableLoadData(TAnyDBRecordset Recordset)
        {
            TStrings Sl;
            String Title;
            String SmallUrl;
            String LargeUrl;
            Integer i;
 
            private String ExtractValue(const String S, const String Subst)
            {
                Integer P;
                P = Pos(Subst + "=", S);
                if (P != 0)
                {
                    Result = Trim(Copy(S, P + Length(Subst) + 1, MaxInt));
                    if (Result.StartsWith("\\\""))
                        Delete(Result, 1, 2);
                    P = Pos("\\\"", Result);
                    if (P != 0)
                        Result = Copy(Result, 1, P - 1);
                }
            }
 
            private String ReplaceSuffix(const String S)
            {
                Result = ChangeFileExt(S, "");
                Result[Length(Result)] = "s";
                Result = Result + ExtractFileExt(S);
            }
 
            private String RemoveSuffix(const String S)
            {
                Result = ChangeFileExt(S, "");
                Delete(Result, Length(Result) - 1, 2);
                Result = Result + ExtractFileExt(S);
            }
 
            private Boolean ExtractPicture(const String S, ref String Title, 
                                           ref String SmallUrl, ref String LargeUrl)
            {
                String Url;
                Title = ExtractValue(S, "title");
                Url = ExtractValue(S, "img src");
                Result = Url != "";
                if (Result)
                {
                    LargeUrl = Url;
                    SmallUrl = Url;
                    Url = ChangeFileExt(Url, "");
                    if (Url.EndsWith("_m"))
                    {
                        SmallUrl = ReplaceSuffix(SmallUrl);
                        LargeUrl = RemoveSuffix(LargeUrl);
                    }
                }
            }
 
            Sl = new TStringList();
            try
            {
                if (HttpGetText("http://api.flickr.com/services/feeds/photos_public.gne?en-us&format=csv", 
                    Sl) && (Sl.Count > 0))
                {
                    for (i = 0; i <= Sl.Count - 1; i++)
                    {
                        if (ExtractPicture(Sl[i], Title, SmallUrl, LargeUrl))
                        {
                            Recordset.Append();
                            Recordset.FieldByName("Title").AsString = Title;
                            Recordset.FieldByName("ThumbUrl").AsString = SmallUrl;
                            Recordset.FieldByName("PictureUrl").AsString = LargeUrl;
                            Recordset.Post();
                        }
                    }
                }
            }
            finally
            {
                Sl.Free();
            }
        }

When you have chosen to create a custom table and defined its data structure, the Morfik development environment allows you to directly input data into it. Any data that is manually entered through the data view of the development environment will be stored as a resource of the application and loaded the when the data set is initialized upon a request for data.

This feature allows for immediate testing of application logic that might be dependent on external information from services which are not yet available. For example, if you are building an application that will display some supporting information that will be extracted from a legacy system through a web services interface, but that interface is not yet available, you can enter some test data into the custom data source and your application will present that information as if it had come from the external service. Once the external service is available the developer can implement the OnLoadData event on the server side of the abstract data source and import the external information on demand. To the application in general there will be no difference between the information that was manually entered in the development environment and that which is returned from the external web service.

• An overview of support for external data sources in Morfik
• Database Connectors
• Native Database Connectivity using Zeos Package