Shameful java 2016-07-21

This is a java snippet I found today.  I’ve changed the method name to protect the guilty party.

 

public String MethodNameChanged(String value) {
   int retVal = 1;
   if (value == null || value.equals(""))
      retVal = 1;
   if (value.equals("" + 30))
      retVal = 1;
   if (value.equals("" + 55))
      retVal = 2;
   return "" + retVal;
}

Questions or problems:

  • To get the string equivalent, the author adds empty string “” to an integer.   This is used in comparison and return value assignment.
  • The original author apparently doesn’t know about the else clause.  So, all of the if statements are evaluated, even if the first is true.
  • If a value of null is passed to this method, the first condition is true, so retVal =1, then the second  condition will also be evaluated and will throw a  null pointer exception.
  • Oh, did I mention that the method isn’t called by anything?  When usage was removed, the original function was left in there.   I’m going to get rid of that ugliness. Anyone interested in admiring it can check prior revisions.
  • Analysis of the logic results allows for a greatly simplified rewrite.

If this method were still used, how about rewriting it as follows?

public String MethodNameChanged(String value) {
    return value!=null && value.equals("55") ? "2" : "1";
}

Of course… it would be a little shorter in Groovy

def MethodNameChanged(value) {
     value?:'' == '55' ? '2' : '1';
}

SpagoBI – Exporting and Importing Report Documents

Step-by-step instructions for Exporting a report document from one server and importing into others.

Steps

  • In the source SpagoBI Environment, go to Import/Export
  • Then, select the documents you want to export.

01-export-select-doc

  • Name the export (20 character limit?)
    • ex:  2015-12-30-scheduled-cand
  • Then, press the export icon

02-export-name-click

  • When the “Download” link appears, click it

03-download

  • Now, You’ve now got a zip file (in downloads) containing the two reports and meta-data about them.  Sign out of POC / DEV
  • This file must have the exact same name when you import into another SpagoBI Environment.  The zip file of the exported document can now be imported into multiple SpagoBI target environments.
  • Next, sign into the Destination SpagoBI server
  • Go to Import / Export again

04-import-export

  • “In Import” , click on “Choose file”

05-import-choose-file

  • Select the file
  • Press the Open button
  • The file name should show. Click on the Import icon

06-import-file-selected-do-import

Roles and associations

  • Visually verify that all roles map correctly
  • Click on next

07-roles-and-assoc

 

Engine Associations

  • For each Engine, carefully select the corresponding engine it maps to in the target spagobi environment
  • Most of the time,   We’re mapping the Birt report engine to the Birt Report Engine
  • Click on Next

08-engine-selection-on-import

Data Source Associations

  • For each Data Source (exported), carefully map it to the data source in the target SpagoBI Environment
  • In most cases, the names should match exactly.  If they don’t match, you may have a problem.   The below screenshot is probably not so helpful.

10-data-source-assoc

Metadata conflicts

  • Review all of the List of Value mappings, Functionality Mappings, and other data
  • Make sure you really want to do this before you proceed
  • Select “Yes”  for “Overwrite the existing meta-data with the exported one?”
    Note: This will overwrite  LOVs, ADs, and these reports in the target tenant. Are you sure you want to do this?

11-import-overwrite-check

  • Click on next
  • Download the log file

12-log-file

  • Keep a copy of the log file and review it to make sure it really deployed what you think it did
  • Go to reports (in spago)
  • Navigate to the reports you just deployed

13-nav-to-report

Interested in Automated Deployments of an Exported Document?

Would you be interested in programmatically deploying an exported SpagoBI document to other SpagoBI environments?

2016-06-16 12_20_21-SpagoBI

SpagoBI 5.2 REST DataSets with JSON Data

Introduction

SpagoBI 5.2 introduces a new dataset type :  RESTful web service datasets . This allows SpagoBI to report on data from sources other than databases with JDBC drivers.

What follows is an end-to-end demonstration of creating and using a REST (JSON) Dataset in SpagoBI 5.2.

External documentation

Description URL
SpagoBI 5.2  NGSI usage of REST Data Sets https://spagobi.readthedocs.io/en/latest/user/NGSI/README/index.html
SpagoBI 5.2 Documentation https://spagobi.readthedocs.io/en/latest/index.html
SpagoBI 5.2 Release Notes https://wiki.spagobi.org/xwiki/bin/view/spagobi_server/release_notes_5_2
JSON Path https://github.com/jayway/JsonPath

Limitations in REST Data Sets

  1. The response of the REST service must be a JSON object.
  2. If the outermost JSON element is an array, it will not work.
    1. There is an assertion that checks that the RESTful call returns a JSON Object.
    2. According to SpagoBI support this is a security feature, not a bug.
    3. However, that means that not all RESTful web services can be used by the SpagoBI REST Dataset. If you’re designing the RESTful web service, that may be fine for you.  If you’re trying to consume a pre-existing RESTful web service, this could be a problem.
  3. “The JSON response must return a flat data structure: SpagoBI cannot handle nested structures”, per SpagoBI Support. (Update 2016-06-22)

Still TBD – DataSet record number limitations

  1. How large or complex of a JSON response can SpagoBI handle?
    1. Per SpagoBI Support, they say they’ve successfully experimented REST Data Sets containing 40,000 records.
    2. Pending experimentation on my part.
      1. I previously attempted record sets with 100 (4MB data), 500 (22 MB data), 1000 (44 MB data), and 5000 records. The JSON was complex and most certainly not flat.
      2. I was able to retrieve the outermost, flat values correctly for the recordsets with 100 and 500 entries.
      3. My experimental recordsets failed processing in setting up the REST Data Set with 1000 or more records.

Example Implementation

Setting up the REST Data Set

Understand the structure of the response for the REST web service you want to use.  Make sure it matches with the limitations above.

Example JSON response

{
        "data" : [
                {"id" : "1", "title" : "this is just a test"},
                {"id" : "2", "title" : "second record"}
        ]
}

In SpagoBI 5.2 (or later)

  • Go to Data Sets
  • Define a new Data Set
  • Set as follows
    • DataSet Type:
      • REST
    • Request Body:
      • if required
    • HTTP Method:
      • Post, Get, Put, or Delete.
      • In this example, use Get
    • JSON Path:
      • Must be a valid JSON Path expression
      • In this example,  we want all JSON objects in the data array
      • Use :  $.data[*]
    • JSON Path Attributes
      • See JSON Path reference
      • See screenshot below
        image2016-6-16 11-34-29

image2016-6-16 11-30-34

 

Click Preview. If everything worked right, you should get back the expected columns and rows.

image2016-6-16 11-35-45

Using the SpagoBI REST Data Set in a Report Document

Now, it is time to use the SpagoBI REST Data set in a Report Document.  For the sake of ease, let’s create a BIRT report in SpagoBI Studio.

In report design view

  • Go to the outline palette
  • Create a new Data Source reference
    • Data Source Type:  select “SpagoBI Server Data Source”
    • Provide the following details
      • Server URL  (Use the dns name or IP of your SpagoBI 5.2 (or later) server
      • user name and password
        image2016-6-16 11-39-50
      • Create a Data Set
        • Data Set Type:   “SpagoBI Server Data Set”
        • Data Set Name:  example: “DS_Z_Rest_04 Data Set”
          image2016-6-16 11-42-37
        • Next
          • Enter the Dataset Label (from SpagoBI).  It must be an exact match with the Data Set label we defined earlier, in SpagoBI server.
            • ex:  DS_Z_Rest_04
              image2016-6-16 11-45-8
          • Now, the output columns should show the columns defined in the data set
            image2016-6-16 11-46-28
  • Now, you’re free to use the data set in the report. In the example below, I’ve defined a 2 column table, associated with the data set. Each column is used.
    image2016-6-16 11-47-35
  • Let’s preview the report
    image2016-6-16 11-47-57
  • Deploy it to SpagoBI Server and run it
    • Other tutorials I’ve provided go into detail on how to deploy and configure reports
  • Run it in SpagoBI Server
    • Default view in SpagoBI execute document
      image2016-6-16 11-49-26
    • Generate a PDFimage2016-6-16 11-50-12

 

 

 

SpagoBI DataSet REST Webservice via Groovy

SpagoBI DataSet REST Webservice via Groovy

UPDATE: THIS IS A GREAT IDEA, EXCEPT IT DOESN’T WORK IN THE FINAL STEP. 
Per SpagoBI Support, Groovy cannot be used in a script-based Dataset to provide rows of data. According to the last support ticket, there is no intent to make that work.

Groovy Data Set to make RESTful web service call and return IP Address related information in SpagoBI’s XML format

Prerequisites :

  1. Upgrade Groovy in SpagoBI and Drop in a copy of the WSLite library
    1. See Upgrading Groovy in SpagoBI
    2. See SpagoBI DataSet SOAP Webservice via Groovy
  2. Restart SpagoBI

Create a new SpagoBI Data Set

DS-tab1

For the Type, Specify Script
For the Script Language, Specify Groovy
Create a parameter : param_ip,  of the type Generic
View the code section below and copy/paste that into your Script section.

DS-tab2-rev2

Groovy code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
import wslite.rest.*
 
def param_ip = parameters['param_ip'] ?: ''
def client = new RESTClient("http://ip-api.com/json/${param_ip.toString()}")
def response = client.get()
 
assert response.statusCode == 200
def str = response.json
 
"""
 
 
 
""".toString()

Press the Save Button

Next…

Now,  Switch to the Preview tab
Enter a Domain or IP address and press the Submit/OK button

ds-tab5a ds-tab5b

SpagoBI DataSet Webservice via Groovy

SpagoBI DataSet Webservice via Groovy

UPDATE: THIS IS A GREAT IDEA, EXCEPT IT DOESN’T WORK IN THE FINAL STEP. 
Per SpagoBI Support:

  • Groovy cannot be used in a script-based Dataset to provide rows of data. According to the last support ticket, there is no intent to make that work.
  • SpagoBI Support also states that the Web-Service Dataset is deprecated and so far refuses to document how to use it.

Note: I have not yet figured out how to get a SpagoBI Data Set (of the type Web Service) to work. However… I’ve figured out a way to make it work by creating a SpagoBI Data Set( type Script) and then using Groovy to make a web service call via Groovy WSLite library.

As part of the POC, I used the publicly available Holiday Web Service to look up Mother’s day for any given year. (Groovy WS-Lite uses it as the simple example).

SO… Round-Trip goes something like this

  1. Report is Requested from SpagoBI , given a parameter of “year”.
  2. SpagoBI executes the Document, delegating to the BIRT report engine.
  3. BIRT report engine executes the Report, passing in the parameter of year
  4. The Report has a SpagoBI Data set reference, which passes the year to the SpagoBI Data Set (back in SpagoBI Core)
  5. SpagoBI Data Set
    • Makes a SOAP call via Groovy and Groovy WS Lite library
    • Parses the response
    • Rewrites the response in the XML format standard for SpagoBI Data Sets, and includes the “requested year” and the response (date for Mother’s day)
    • The Report retrieves the data from the SpagoBI Data Set
    • Report renders showing the requested year and the date on which Mother’s Day occurs for that year

Dependencies:

Groovy WSLite 2.0
Groovy-based Web-Service library which supports SOAP and RESTful calls
Versions

  • 2.0 Beta . Requires Groovy 2.3 or later. Worked fine for SOAP. Failed for RESTful call
  • 1.0 / 0.7.1 : Requires Groovy 1.7.6 or later.  Worked fine for SOAP and REST

Groovy Language
We’re using Groovy WS Lite, required a newer version of Groovy than that which shipped with SpagoBI 5.0 (Groovy 1.5.6)
So, it was necessary to replace the Groovy-all library jar with a later version and add the Groovy WS Lite Jar
Download : groovy-all-2.4.3.jar

Steps

  • See instructions for Upgrading Groovy and Installing Groovy WS Lite
  • Log into Spagobi
  • Create a new data set
  • Create a new data set
  • Set the Data Set label
  • Set the Data Set Type (on the Type tab) to Script
  • Set the Language for the Script to Groovy
  • Setup a parameter named “param_year”, with the type of “Generic”
  • Add the script code (below)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
import wslite.soap.*
 
 
def param_year = parameters['param_year']
 
// When SpagoBI Studio inquires with SpagoBI Server for the definition of the data set, 
// it will not work because it doesn't pass a default parameter
// Get around this by specifying a parameter value to use when none is provided.
 
if (param_year ==null) {
   param_year  = "2015"
}
 
def client = new wslite.soap.SOAPClient('http://www.holidaywebservice.com/Holidays/US/Dates/USHolidayDates.asmx')
def response = client.send(SOAPAction:'http://www.27seconds.com/Holidays/US/Dates/GetMothersDay') {
    body {
        GetMothersDay('xmlns':'http://www.27seconds.com/Holidays/US/Dates/') {
            year("${param_year.toInteger()}")
        }
    }
}
//assert "2011-05-08T00:00:00" == response.GetMothersDayResponse.GetMothersDayResult.text()
assert 200 == response.httpResponse.statusCode
assert "ASP.NET" == response.httpResponse.headers['X-Powered-By']
 
// Remember Java doesn't know how to interpret a Groovy GString. The resulting output must be cast to a Java String
// If you leave it as a GString preview in SpagoBI Server will work  but Studio will be unable to attach to the data set.
 
"""
 
 
 
""".toString()
  • Preview in SpagoBI (Server) providing a year. Press the preview button and you should get the response back from the Holiday Web Service
    Preview Groovy-WS Data Set Prompt

    Preview Groovy-WS Data Set Prompt

    SpagoBI Groovy-WS Preview Result

    SpagoBI Groovy-WS Preview Result

  • The next post will cover consuming that SpagoBI Data Set in a BIRT report