Power BI Embedded Walk Through Part 3 of 3

Power BI Embedded Walk Through Part 1 of 3
Power BI Embedded Walk Through Part 2 of 3

In the previous 2 blog articles, I have shown how to create a Power BI Embedded       Workspace collection, import a Power BI desktop file and get generated embed details.

To recap from the last post, I obtained the embed Url Embed Url: https://embedded.powerbi.com/appTokenReportEmbed?reportId=47e3e117-65b9-4bb4-8283-98525e5b7c59
Embed Token:
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ2ZXIiOiIwLjIuMCIsInR5cGUiOi
JlbWJlZCIsIndjbiI6InJrcGJpIiwid2lkIjoiMjY0MTJlNGQtNmQ0YS00ZTE1LTk3MzEtOTMxNzdkNT
U1ODNmIiwicmlkIjoiNDdlM2UxMTctNjViOS00YmI0LTgyODMtOTg1MjVlNWI3YzU5Iiwic2NwIjoiUm
Vwb3J0LlJlYWRXcml0ZSIsImlzcyI6IlBvd2VyQklTREsiLCJhdWQiOiJodHRwczovL2FuYWx5c2lzLn
dpbmRvd3MubmV0L3Bvd2VyYmkvYXBpIiwiZXhwIjoxNDk0NDY5ODY5LCJuYmYiOjE0OTQ0NjYyNjl9.n
ZW5lcop9VNLf1mWzc3-QUN2L1WaMe_b5bI5SQ6CojU

Create or use an existing web application. For myself, I created a new ASP .NET web application with Single Page Application template
Power BI Embedded Walk Through Part 3 of 3-1

I add the Power BI JavaScript nugget package into my project.
Power BI Embedded Walk Through Part 3 of 3-2
Thus, I see the powerbi.js file added
Power BI Embedded Walk Through Part 3 of 3-3

The following places to put the code.
In App_Start\BundleConfig.cs, add the Power BI script reference

bundles.Add(newScriptBundle("~/bundles/app").Include(
"~/Scripts/sammy-{version}.js",
"~/Scripts/app/common.js",
"~/Scripts/powerbi.js",
"~/Scripts/app/app.datamodel.js",
"~/Scripts/app/app.viewmodel.js",
"~/Scripts/app/home.viewmodel.js",
"~/Scripts/app/_run.js"));

In Scripts\app\home.viewmodel.js, add the PowerBI embedded JavaScript code snippet. Note the configuration needed for the accessToken, embedUrl, and id. You can get this in my post of part 2 of 3. As for the accessToken, it has a short expiry. In the ProvisionSample App, there is a call to createEmbedToken, and you can pass an expiry date to control the lifetime of the token. As for some refresh token mechanism, I have yet figure out how that would happen.

As or KnockoutJS conventions, I could have not used JQuery on the reportContainer div and did proper binding.

functionHomeViewModel(app, dataModel) {
var self = this;
    self.myHometown = ko.observable("");

    Sammy(function () {
// Removed function code for brevity
    });

var models = window['powerbi-client'].models;

var PBIconfig = {
        type: 'report',
accessToken: 'eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ2ZXIiOiIwLjIuMCIsInR5cGUiOiJlbWJlZCIsIndjbiI6InJrcGJpIiwid2lkIjoiMjY0MTJlNGQtNmQ0YS00ZTE1LTk3MzEtOTMxNzdkNTU1ODNmIiwicmlkIjoiMDYxODlkYmQtYWRhZS00YWRiLTljMDQtZTNkMGNkZmIxZmQ3Iiwic2NwIjoiUmVwb3J0LlJlYWRXcml0ZSIsImlzcyI6IlBvd2VyQklTREsiLCJhdWQiOiJodHRwczovL2FuYWx5c2lzLndpbmRvd3MubmV0L3Bvd2VyYmkvYXBpIiwiZXhwIjoxNDk0NjQwMzM3LCJuYmYiOjE0OTQ1NTM5Mzd9.VroIsYUxVDzbgHl0HyJ2sJTPRWWtQ6wjzZkrRXP8SmQ',
embedUrl: 'https://embedded.powerbi.com/appTokenReportEmbed?reportId=06189dbd-adae-4adb-9c04-e3d0cdfb1fd7',
id: '06189dbd-adae-4adb-9c04-e3d0cdfb1fd7',
        permissions: models.Permissions.All /*gives maximum permissions*/,
        viewMode: models.ViewMode.Edit,
        settings: {
            filterPaneEnabled: true,
            navContentPaneEnabled: true
        }
    };

var reportContainer = $('#reportContainer')[0];
var report = powerbi.embed(reportContainer, PBIconfig);
// Report.off removes a given event handler if it exists.
    report.off("loaded");
// Report.on will add an event handler which prints to Log window.
    report.on("loaded", function () {
        Log.logText("Loaded");
    });
    report.off("error");
    report.on("error", function (event) {
        Log.log(event.detail);
    });
    report.off("saved");
    report.on("saved", function (event) {
        Log.log(event.detail);
if (event.detail.saveAs) {
            Log.logText('In order to interact with the new report, create a new token and load the new report');
        }
    });

return self;
}

Go to Views\Home\_Home.cshtml and the div container for the iFrame to display the Power BI report
<divid="reportContainer"style="height:600px"></div>

Run the application in preferably Chrome browser
Power BI Embedded Walk Through Part 3 of 3-4

Since this is in Edit mode, the user can interactively edit the chart, add to it as if you were in Power BI Service or Power BI desktop. Also, the ability save or save as. I find this quite neat and serves DIY analytical scenarios.

Going back to Azure Portal, since a user had started a session with this report, you can see the one embed session.
Power BI Embedded Walk Through Part 3 of 3-5

Final Remarks
My experience to learn about Power BI Embedded was quite challenging and time-consuming as each online documentation told part of the entire developer story. I hope I have shown a good end to end walkthrough to understand what is involved.


Building a Spark Application for HDInsight using IntelliJ Part 2 of 2

In continuation from my blog article Building a Spark Application for HDInsight using IntelliJ Part 1 of 2 which outlines my experience in installing IntelliJ, other dependent SDKs and creating an HDInsight project.

To add some code, right click src, create Scala Class
Building a Spark Application for HDInsight using IntelliJ Part 1 of 2-1
Building a Spark Application for HDInsight using IntelliJ Part 1 of 2-2

Project folders and MainApp
Building a Spark Application for HDInsight using IntelliJ Part 1 of 2-3

Scala code:

import org.apache.spark.SparkConf
import org.apache.spark.SparkContext
import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.SQLContext

object MainApp{
def main (arg: Array[String]): Unit = {
val conf = new SparkConf().setAppName("MainApp")
val sc = new SparkContext(conf)

val rdd = sc.textFile("adl://rkbigdata.azuredatalakestore.net/MyDatasets/Crimes_-_2001_to_present_pg2.csv")
//find the rows where primary type == THEFT
val rdd1 =  rdd.filter(s => s.split(",")(5) == "THEFT")

val spark = SparkSession.builder().appName("Spark SQL basic").enableHiveSupport().getOrCreate()

    spark.sql("USE usdata")
val crimesDF = spark.sql("SELECT * FROM CRIMES WHERE primarytype == 'NARCOTICS'")

    // save data frame of results into an existing or non-existing hive table.
crimesDF.write.mode("overwrite").saveAsTable("crimebytype_NARCOTICS")
  }

}

Logic

  1. Read from csv file in Azure Data Lake Store into RDD
  2. Filter RDD for rows where primary type field is “THEFT”
  3. Set Hive Database to usdata (from default database)
  4. Query Hive table CRIMES for rows primary type field is “THEFT”
  5. Save data frame into new or existing crimebytype_NARCOTICS hive table.

Begin to setup IntelliJ to submit application to HDInsight

Before using Azure Explorer, I encountered an issue where signing in resulted in an error and it kept prompting me to enter the credentials. Sorry I didn’t capture the error message. And so, I was led to disable Android Support by checking it off.
Building a Spark Application for HDInsight using IntelliJ Part 1 of 2-4

Click Ok.
Sign into Azure via Azure Explorer
Building a Spark Application for HDInsight using IntelliJ Part 1 of 2-5
Select Interactive
Enter credentials

Building a Spark Application for HDInsight using IntelliJ Part 1 of 2-6

See the Azure resources display

Building a Spark Application for HDInsight using IntelliJ Part 1 of 2-7
Right click the Project and click on Submit Spark Application to HDInsight

Building a Spark Application for HDInsight using IntelliJ Part 1 of 2-8
Set Main class name to MainApp
Building a Spark Application for HDInsight using IntelliJ Part 1 of 2-9
HDInsight Spark Submission window
Confirm success

Building a Spark Application for HDInsight using IntelliJ Part 1 of 2-10
Go to Ambari Hive View to query the hive table created from the spark application.
Building a Spark Application for HDInsight using IntelliJ Part 1 of 2-11
From Jupyter notebook, I query the same hive table.

Building a Spark Application for HDInsight using IntelliJ Part 1 of 2-12

I have shown a walk through of setting up the development tooling and building a simple spark application and run against HDInsight Spark 2.0 Cluster.


Azure Data Lake Analytics: U-SQL C# Programmability

One of the neat features of U-SQL for .NET developers are that you can write C# code for further extensibility and be applying custom logic. This is useful for scenarios for extensive ETL processes where you must take raw data, clean, scrub, transform to meet business requirements for further analytical processing and BI reporting. The alternative languages for Big Data analytics are languages such as Python, R, Scala and Pig. They are more popular in the Big Data community and serve their intended use cases very well. And is favoured by various type of data developers. For the .NET developer who wants to generally start out, then U-SQL is a good choice. After some time, one can advance to those other ‘big data’ languages.

U-SQL programmability guide

My U-SQL with C# code involves counting the frequency of words that appear in a text-based field. The purpose of this code is to test a hypothesis whether getting the most frequent words in a text field can help determine what the text field is about. For example, if the job description field has top words such as ‘developer’, ‘javascript’, ‘json’, then perhaps it gives some indication that the job is a technical job. Let’s continue to explore.

@jobpostings variable represents the data from a file extraction. Let’s process it further with C# code-behind to count the occurrence of each word in the jobDescription field.

@jobpostings_Parsed =
     PROCESS @jobpostings
     PRODUCE jobtitle string,
        company string,
        city string,
        state string,
        country string,
        date string,
        url string,
        latitude float,
        longitude float,
        jobDescription string,
        salaryRate string,
        jobType string,
        hourlyRate double?,
        salary double?,
        compensationType string,
        jobDescriptionTopWords string,
        jobTitleTopWords string
     USING new JobPostings.USql.JobPostingsParser();

For the C# code-behind, I am using the User-defined processor which allows custom logic to be applied on each row. For details read https://docs.microsoft.com/en-us/azure/data-lake-analytics/data-lake-analytics-u-sql-programmability-guide#use-user-defined-processors

Azure Data Lake Analytics- U-SQL C# Programmability - 1

using Microsoft.Analytics.Interfaces;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text.RegularExpressions;

namespace JobPostings.USql
{
    public class JobPostingsParser : IProcessor
    {
	 // Dictionary to tally each word and its occurence
        Dictionary<string, int> jobDescriptionFreqWords = new Dictionary<string, int>();

	 // Process each row
        public override IRow Process(IRow input, IUpdatableRow output)
        {
            string jobDescription = input.Get<string>("jobDescription");

                if (!String.IsNullOrEmpty(jobDescription))
                {
                    Dictionary<string, int> <strong>jobDescriptionFreqWords</strong> = GetMostFrequentWords(jobDescription);
                    List<KeyValuePair<string, int>> myList = jobDescriptionFreqWords.ToList();

			// Sort by descending
                    myList.Sort(
                        delegate (KeyValuePair<string, int> pair1,
                        KeyValuePair<string, int> pair2)
                        {
                            return pair2.Value.CompareTo(pair1.Value);
                        }
                    );

			// To output the three most frequent words and their word count as a text string into a new field called jobDescriptionTopWords
                    if (myList.Count >= 3)
                        output.Set<string>("jobDescriptionTopWords", myList[0] + ", " + myList[1] + ", " + myList[2]);
                }

// Need to explicitly output each field even though there is no processing on them.
            output.Set<string>(0, input.Get<string>("jobtitle"));
            output.Set<string>(1, input.Get<string>("company"));
            output.Set<string>(2, input.Get<string>("city"));
            output.Set<string>(3, input.Get<string>("state"));
            output.Set<string>(4, input.Get<string>("country"));
            output.Set<string>(6, input.Get<string>("date"));
            output.Set<string>(8, input.Get<string>("url"));
            output.Set<float>(9, input.Get<float>("latitude"));
            output.Set<float>(10, input.Get<float>("longitude"));
            output.Set<string>(16, input.Get<string>("jobDescription"));
            output.Set<string>(17, input.Get<string>("salaryRate"));
            output.Set<string>(18, input.Get<string>("jobType"));

            return output.AsReadOnly();
        }
	// Helper method to tally frequency of words for a given text field
private Dictionary<string, int> GetMostFrequentWords(string text)
        {
            Dictionary<string, int> wordCount = new Dictionary<string, int>();
            Dictionary<string, int> stopWords = new Dictionary<string, int>() {
                { "a", 0 }, { "able", 0 }, { "about", 0 }, …<omitted for brevity> …
            };

            char[] delimiters = new char[] { ' ', '\r', '\n', ',', '.', ':' };
            string[] words = text.Split(delimiters, StringSplitOptions.RemoveEmptyEntries);
            string w;

            foreach (string word in words)
            {
                w = word.ToLower();
                if (stopWords.ContainsKey(w))
                    continue;

                if (wordCount.ContainsKey(w))
                    wordCount[w]++;
                else
                    wordCount.Add(w, 1);
            }

            return wordCount;
        }

To summarize the logic:

  1. Read each row
    1. Get the jobDescription field value
    2. Call custom GetMostFrequentWords to return a dictionary of word and its frequency
      1. GetMostFrequentWords simply keeps a tally of each word encountered for that row but ignores a predefined static list of stop words. The list of stop words is from an arbitrary internet source.
      2. Sort by descending the tally of frequent words
      3. Construct a text string of 3 key/value pairs of the top most frequent words.
      4. Output this text string into a new field called jobDescriptionTopWords

After running the u-sql script and doing a file preview, the following shows the jobDescriptionTopWords field and the resulting text string to show the top 3 words.

Azure Data Lake Analytics- U-SQL C# Programmability - 2     Azure Data Lake Analytics- U-SQL C# Programmability - 3

So, going back to my hypothesis of whether displaying most frequent words gives an indication of the type of job, I think there is still some ambiguity. For example, the first row had most frequent words of visit, flexible, anytime. I wouldn’t be able to guess what type of job this would be. The associated job title is Merchandiser. Looking at the row with words heat, machine and sewing, I would have guessed that this job did indeed involve operating a sewing machine. My conclusion is that top word counting in a text field with many words can help, but not significantly. To be more productive, use other text analysis services to extract more meaning. In this article, I have shown the use of U-SQL C# programmability with using the user-defined processor to count the frequency of words in a row’s text field.


SharePoint 2013 Custom AJAX filters on List View Web Parts

Requirement:
Implement ‘exact match’ filter against the list view web part on text values and drop down values using AJAX for great user performance. An alternative to filtering in a fly out menu of the list view web part’s column values.ajaxfilter1
Filter by ‘Title’
ajaxfilter2

Filter by ‘Task Status’
ajaxfilter3

Technique:

I am essentially emulating the column based filters that are out-of-the-box functionality of the SharePoint 2013 List view web parts. I do this by tracking down how the filtering is executed in the SharePoint inplview.js code. Then wrap the filtering JavaScript logic through custom text boxes.

ajaxfilter4

You can see the URL query string parameters reflect the filtering
/SitePages/Tickets.aspx#InplviewHashdba27ef6-7bff-4c0f-8ec1-b3e3e18fb0b8=FilterField1=LinkTitle-FilterValue1=Printer broken

For large lists (i.e. > 5,000 items), keep in mind to index the columns you want to filter by. You can find the setting in the List Settings page and find a link ‘Indexed Columns’

<table style="" border="0">
    <tr>
        <td>Title:</td>
        <td><input type="text" id="title" onblur="TitleFilter(); return false;" /></td>
        <td>
            <button onclick="TitleFilter(); return false;">Filter by Title</button>
        </td>
    </tr>
    <tr>
        <td>
            Task Status:
        </td>
        <td>
            <input type="text" id="taskStatus" onblur="TaskStatusFilter(); return false;" />
        </td>
        <td>
            <button onclick="TaskStatusFilter(); return false;">Filter by Task Status</button>
        </td>
    </tr>
    <tr>
        <td></td>
        <td></td>
        <td>
            <button onclick="TitleAndTaskStatusFilter(); return false;">Filter by Title AND Task Status</button>
        </td>
    </tr>
</table>
 

<script language='javascript'>
       function MyRefreshPageToEx(lvTableID, url, bForceSubmit) {

        // Hardcode reference to list view <table> Id attribute
        var tblv = document.getElementById("{BE9A90D1-BF5D-401B-8A9B-C44CCCD14241}-{DBA27EF6-7BFF-4C0F-8EC1-B3E3E18FB0B8}");
        var clvp = CLVPFromCtx(tblv);

        if (clvp != null && clvp.ctx.IsClientRendering) {
            clvp.RefreshPaging(url);
            clvp.ctx.queryString = url;
            if ((typeof clvp.ctx.operationType == "undefined" || clvp.ctx.operationType == SPListOperationType.Default) && Boolean(clvp.ctx.ListData)) {
                var fromPage = clvp.ctx.ListData.FirstRow - 1;
                var toPage = Number(GetUrlKeyValue("PageFirstRow", false, url));

                if (!isNaN(fromPage) && !isNaN(toPage) && fromPage != toPage)
                    fromPage < toPage ? (clvp.ctx.operationType = SPListOperationType.PagingRight) : (clvp.ctx.operationType = SPListOperationType.PagingLeft);
            }
        }
        else {
            SubmitFormPost(url, bForceSubmit);
        }
    }

    function TitleFilter() {
        var title = $('#title').val()
        var url;
        if (title) {
            url = "?List={BE9A90D1-BF5D-401B-8A9B-C44CCCD14241}&View={DBA27EF6-7BFF-4C0F-8EC1-B3E3E18FB0B8}&FilterField1=LinkTitle&FilterValue1=" + title
        }
        inplview.MyRefreshPage = MyRefreshPageToEx;
        inplview.MyRefreshPage(null, url, null);
    }

    function TaskStatusFilter() {
        var taskStatus = $('#taskStatus').val()
        var url;
        if (taskStatus) {
            url = "?List={BE9A90D1-BF5D-401B-8A9B-C44CCCD14241}&View={1F1B1E55-992B-4A03-98B2-9E5D34916611}&FilterField1=TaskStatus&FilterValue1=" + taskStatus
        }
        inplview.MyRefreshPage = MyRefreshPageToEx;
        inplview.MyRefreshPage(null, url, null);
    }

    function TitleAndTaskStatusFilter() {
        var title = $('#title').val()
        var taskStatus = $('#taskStatus').val()

        // HARD Code: query string to reference List GUID and View ID
        var url = "?List={BE9A90D1-BF5D-401B-8A9B-C44CCCD14241}&View={DBA27EF6-7BFF-4C0F-8EC1-B3E3E18FB0B8}&ViewCount=1&IsXslView=TRUE&IsCSR=TRUE&";
        if (title && taskStatus) {
          url = url + "FilterField1=LinkTitle&FilterValue1=" + title + "&FilterField2=TaskStatus&FilterValue2=" + taskStatus
        } else if (title && !taskStatus) {
          url = url + "FilterField1=LinkTitle&FilterValue1=" + title
        } else if (!title && taskStatus) {
          url = url + "FilterField1=TaskStatus&FilterValue1=" + taskStatus
        }
        inplview.MyRefreshPage = MyRefreshPageToEx;
        inplview.MyRefreshPage(null, url, null);
    }
</script>
 

There is some hard coding in the custom JavaScript code making references to the list view web part such as the table Id, list GUID and view GUID.

Find the table element Id as follows.
ajaxfilter5

Limitations

  • Inplview.js filtering only supports exact match of values and not partial match
  • OR operand is not supported in the inpview.js from what I can see.
  • This custom approach is a bit of “hack”. Since it is client side, there is no impact to the SharePoint farm on the server side. Nevertheless, follow any development guidelines in your team. At the least, regression testing is recommended on SharePoint product patching and upgrades for any breaking changes.

The solution fit

I think the better solution fit is where the user already knows exactly what to filter on, but doesn’t want to scroll a largest list of values through the OOTB column filters every single time. For example, filtering by exact name or a small number.

SharePoint 2013 Custom List View Web Part Refresh

In the SharePoint 2013 list view web part, the paging, filtering and sorting functionality is driven by the inplview.js file. This is supported by AJAX calls against a RESTful service inplview.aspx. I am able to extend the JavaScript inplview object to implement client side refresh as data is updated on the server.
refresh1

To configure the automatic refresh by OOTB (out-of-the-box) configuration, go to the Web Part Properties: 

refresh2

 

What I find unattractive of this approach is that it is using the older AJAX UpdatePanel introduced in ASP .NET 2.0.

  • There is at least 23kb transferred for each refresh which is a relatively heavier in bandwidth.
    refresh3
  • The http response time is longer.
  • Using IE 8 browser, there is a memory leak such that memory utilization by the browser maxes out.
  • The out of the paging, filtering and sorting with the AJAX options do not work well together. For example, the AJAX refresh option sometimes shows no rows even after repeated combination of filtering and manual OOTB AJAX refresh. To resolve, refresh the page without any query string parameters.

The applicable business scenario or requirement is when frequently added and updated list items by multiple users are to be seen updated in the browser without user requiring to manually refresh the page. 

In SharePoint 2013, when the user interacts with the list view web part for activities such as paging, column filtering and sorting, there is an AJAX request making a RESTful call to inplview.aspx.
refresh4 Compared to the OOTB refresh, the inplview.aspx REST calls are significantly lighter in bandwidth and has faster response time.

How inplview.js Works

The javascript code that supports the list view web part paging, sorting and filtering is in the inplview.js file

The sequence of calls are made as follows:

  1. Browser
    1. User clicks on list view web part to page, sort and filter
    2. invoke objects in various SharePoint .js files
    3. invoke objects inplview.js
  2. Server
    1. inplview.aspx that is a RESTful http service
    2. HTTP response back to browser with only JSON formatted row data.

Using the chrome JavaScript debugger against inplview.js, I have identified the following sequence of function calls. To support the refresh, I have “forked” the code with my own custom functions.

Original OOTB sequence Customized “forked” sequence
RestoreAllClvpsNavigation Refresh_RestoreAllClvpsNavigation
EnumCLVPs(RestoreClvpNavigation) EnumCLVPs(Refresh_RestoreClvpNavigation)
RestoreClvpNavigation(clvp) <skip>
clvp.RestoreNavigation(); <skip>
CLVPRestoreNavigation() Refresh_RestoreClvpNavigation(clvp)

The Implementation

There is no explicit data refresh support in the inplview javascript code so I have injected custom code that extends the inplview javascript object to have supporting functions to implement refresh.

// automatic refresh based on interval
function autoRefresh()
{
 window.setInterval(listViewRefresh, 2000); // 20 seconds
}

// refresh all list view web parts on the page
function listViewRefresh() {
 $('#lblMessage').text('refreshed ').fadeIn("slow").fadeOut("slow"); // debugging
 inplview.MyRestoreAllClvpsNavigation = MyRestoreAllClvpsNavigation;
 inplview.MyRestoreAllClvpsNavigation();
}

// Enumerate list view web parts
function MyRestoreAllClvpsNavigation()
{
 EnumCLVPs(MyCLVPRestoreNavigation);
}

// refresh referencing list view web part
function MyCLVPRestoreNavigation(clvp) {
 var strHash = ajaxNavigate.getParam("InplviewHash" + clvp.WebPartId());
 if (strHash == null)
 strHash = '';

 var strInpl = '?' + DecodeHashAsQueryString(strHash);
 var strShowInGrid = GetUrlKeyValue("ShowInGrid", true, strInpl);

 if (strShowInGrid == "True") {
 InitGridFromView(clvp.ctx.view, true);
 }
 else if (clvp.ctx.inGridMode) {
 ExitGrid(clvp.ctx.view, true);
 }
 clvp.strHash = strHash;
 clvp.fRestore = true;
 var curRootFolder = GetRootFolder2(this);

 if (curRootFolder != null)
 strInpl = SetUrlKeyValue("RootFolder", unescapeProperly(curRootFolder), true, strInpl);
 clvp.RefreshPagingEx(strInpl, true, null);

 }

$(document).ready(function() {
 autoRefresh();
});

refresh5

Benefits

  1. Overcome IE8 browser memory leak issue
  2. Noticeable performance improvement over OOTB Ajax option
  3. The ability to extend and customize the client side rendering of the list view web part functionality

Caveats

  1. Upgrades and patches:
    Since this is extending from product JavaScript code, in the event of upgrades or patching, there could breaking changes.
    To mitigate this risk, remember to regression test after a product upgrade and patch.

Supportability

  1. Still supports general list view paging, column filtering and sorting.
  2. Exception: column filtering and sorting is not supported on lookup columns as it is already with OOTB.
  3. Ability to refresh all list view web parts that exist on the same page.

I’ll have a another blog post to add filtering based on user input by customizing the inpvliew.js

Visual Studio 2012 SharePoint 2013 List and Content Type Designer: Hands-on Observations

After spending some time on developing a SP Hosted App, I would like to share my insights and observations with the new Office Developer tools for SharePoint 2013

Prerequisite: Web Platform Installer > Microsoft Office Developer Tools for Visual Studio 2012

Let’s walk through my observations from site columns, to content types and to list definitions.

Site Columns

  • There is no visual designer. Content types reference site columns and so having a site column designer would make much sense. It is unfortunate that one is absent.
  • Therefore, creating site columns is manual through the elements.xml file

VSContentTypeDesigner-1

Field Element Schema
http://msdn.microsoft.com/en-us/library/ms437580.aspx

Content Type

  • There is a new visual designerVSContentTypeDesigner-2VSContentTypeDesigner-3
  • Add existing OOTB site columns
  • Add existing custom site columns that were manually created.VSContentTypeDesigner-4
  • Remember, you cannot create custom site columns in the content type designer.
  • Limitation: Column ordering is manual through the elements.xml definition files. In other words, you cannot reorder site columns in the conten type designer.
  • Gotcha: Change in custom site column definition does not automatically reflect changes in content type. For example, changing site column name will break the referring content type.
    Therefore, you must remove the custom site column and then add updated site column.

List Definition and Instance

  • There is new visual designer
    Able to set List (template) name, url and description.
  • Content Types. Add existing OOTB content type or custom content type created in the VS projectVSContentTypeDesigner-5
  • Columns
    Add existing OOTB site columns or
    existing custom site columns in the VS project
    VSContentTypeDesigner-6Add list level columns manually
    VSContentTypeDesigner-7
  • Limitation: When creating a new custom column that are more complex in nature such as choice or person, there is no ability to configure the selectable choices. I would think you would have to revert to the xml files and fill in the rest of the xml manually. Having such a site column designer would have been very convenient.
  • When creating a list column, it automatically creates a list content type called “ListFieldsContentType” if one does not exist. This becomes the default. Any further site columns (OOTB or custom) added will be automatically added to this ListFieldsContentType content type.
    VSContentTypeDesigner-8
  • Note: Changes in content types referenced by the list definition are not automatically reflected. This is the same scenario mentioned above with site columns referenced by content types. So after you change a referenced content type, you must remove and then add back the updated content types to the list definition.
  • Set default content type so that the list’s ‘new’ button refers to the content type desired.
  • Limitation: List column ordering is manual through the xml definition files.
  • Not able to add predefined data rows in List Instance
    In ListInstance elements.xml, I added
    <DataRows>
    <Row>
    ….
    </Row>
    </DataRows>
    This resulted in a deployment error. Maybe I am missing something, but I was unsuccessful.
  • Views
    Able to specify which columns appear in the view and their order, row limit, read only, but nothing more than that. For example, no sorting ability, grouping, filtering, etc.
    VSContentTypeDesigner-9

After deploying to site collection:

VSContentTypeDesigner-10

Deployment Error: Failed to install app

Possible causes:

  • The xml schema files are malformed or have a syntax issue. Visual Studio can detect and highlight some of these in the Error pane.
  • A custom element definition is in conflict with an element already existing in the SharePoint farm.
    For example, deploying a list, content type or site column that has the same internal name as an existing artefact. Duplication error.

Troubleshooting:

The way I troubleshoot, is to remove snippets of xml definition and re-deploy to isolate trouble area.

I have looked in the 15 ULS Logs to find any detailed error messages and did not recognize anything helpful.

Conclusion:

The designers are a welcomed feature in Visual Studio 2012 for SharePoint developers. However, more can be achieved with site columns and handling changes to referenced elements. This is probably a large effort, since there are so many types of site columns with varying complexities such as ‘People and Groups’ and ‘Managed Metadata”. Nevertheless, it is one step better in getting through the xml hell.

Appreciate any comments and feedback.

References:

Walkthrough: Create a Site Column, Content Type, and List for SharePoint
http://msdn.microsoft.com/en-us/library/ee231593.aspx

Common SharePoint Web Part Errors

The following are some of the common web part errors that I just wanted to summarize and provide the general solution.

Error Message:

A Web Part or Web Form Control on this Page cannot be displayed or imported. The type could not be found or it is not registered as safe.

Image

OR

A Web Part or Web Form Control on this Page cannot be displayed or imported. The type DateTimeWebPart.DateTimeWebPart.DateTimeWebPart, DateTimeWebPart, Version=1.0.0.0, Culture=neutral, PublicKeyToken=2bfd9e5fd3b67b15 could not be found or it is not registered as safe.

OR

An error occurred when previewing the Web Part.

Ways to reproduce error

  • Preview web part by Site Settings > Web parts (Galleries) > Click web part list item; OR
  • Insert web part to a page; OR
  • View a page that previously inserted the web part.

The files in question

Image

1) .cs

Image

2) .webpart

Image

3) .spdata
(hidden – must ‘show all files’ at the Visual Studio project level)

Image

OR

Right Click <WebPart> Project item > Properties > Click Safe Control Entries

Image

4) Web.config
(IIS virtual directory, e.g. C:\inetpub\wwwroot\wss\VirtualDirectories\80)

The safe control entry is automated in deployment declared in the .spdata file.

<SafeControls>
<SafeControl Assembly=”DateTimeWebPart, Version=1.0.0.0, Culture=neutral, PublicKeyToken=2bfd9e5fd3b67b15″ Namespace=”DateTimeWebPart.DateTimeWebPart” TypeName=”*” Safe=”True” />
</SafeControls>

5) Assembly / DLL file missing  in the Global Assembly Cache for GAC deployment (c:\windows\assembly) or virtual directory bin folder in web app deployment.

Resolution

Make sure the Namespace and Type Name are consistent across all files where indicated. Also with matching case sensitivity.

Verify web.config file and assembly in GAC or virtual directory bin folder in post deployment.

Background

What does it mean by ‘type could not be found’? What is the type?

The type in question is the class inheriting from WebPart class. In this case, it is DateTimeWebPart.DateTimeWebPart.DateTimeWebPart. Excuse my naming convention. When the .NET run time is creating an instance of the web part, the run time can not find the type; either because the assembly is missing, inaccurately referenced or misnamed.

What does it mean by ‘Is not registered as safe’?

Given that the type can be found and the type is registered in the web.config in the <SafeControls> xml node.

The safe control entries mark a web control (instantiated in the web part) such that an untrusted user has access to the web control. Or another way to look at it is that the web application trusts the web control.

SharePoint 2013 Hosted App Development Presentation Deck

I had the pleasure to speak at the January 2013 Toronto SharePoint User Group meeting.

Topic: SharePoint 2013 Hosted App – Deep Dive Implementation

The last few months of I have been working on a photo slider app displaying a photo, title and caption. I wanted to learn the technologies, architecture and tooling. My presentation is to explain my experience and knowledge in making this app. At the same time, I got some great tips and feedback from the group.

Issue: Visual Studio 2012 > “Unable to launch the IIS Express Web Server”

Steps to Problem/Issue

  • I installed Visual Studio 2012 RTM.
  • Created an MVC 4 project template in Visual Studio 2012
  • Without any code changes, I hit F5 debug.
  • Issue: I get a pop up message “Unable to launch the IIS Express Web Serve

After much searching and many suggest re-install Visual Studio 2012, but it takes a long time (>20 mins) and I’m pretty stubborn at times to find root cause.

I found myself looking at the applicationhost.config file located at
C:\Users\<username>\Documents\IISExpress\config

Look for <applicationPools> node.

I updated managedRuntimeVersion=”4.0″ to “4.0.30319”

Example:
<applicationPools>
<add name=”Clr4IntegratedAppPool” managedRuntimeVersion=”v4.0.30319″ managedPipelineMode=”Integrated”     CLRConfigFile=”%IIS_USER_HOME%\config\aspnet.config” autoStart=”true” />

Back to Visual Studio 2012 Project Solution, I hit F5 Debug and IE loaded with the App!

Roy Kim