Thursday, February 08, 2024

Alliance Conference 2024


Alliance 2024 is just a few weeks away, and I look forward to meeting with friends and colleagues from the PeopleSoft community. I hope to see you in the exhibition hall and in customer sessions.

I am excited to present the following sessions:

  • Getting the Most out of PeopleSoft PeopleTools: Tips and Techniques on March 5 at 2 PM in North 122 ABC.
  • Get Current Faster by Isolating Customizations on March 6 at 2 PM North 122 ABC.

See you there!

Thursday, January 18, 2024

Generating LARGE JSON Files

The PeopleCode native JsonObject and JsonArray classes allow us to create JSON structures as in-memory representations. But what if you need to generate a really LARGE JSON structure? An in-memory JSON Array may consume more memory than you can reasonably allow. Fortunately, PeopleTools includes the Jakarta JSON library, which allows us to write a JSON structure to a stream during construction.

The following code snippet demonstrates creating 10 million JSON objects in an array without any change in memory consumption. The generated file was 2.5 GB in size, but my memory utilization didn't change the entire time the program ran.

Local JavaObject &Json = GetJavaClass("jakarta.json.Json");
Local JavaObject &writer = CreateJavaObject("java.io.FileWriter", "C:\temp\users-big.json");

Local JavaObject &gen = &Json.createGenerator(&writer);

Local number &iteration = 1;

REM ** 10 million iterations;
Local number &maxIterations = 10000000;

&gen.writeStartArray();

For &iteration = 1 To &maxIterations
   
   REM ** start person/user object;
   &gen.writeStartObject();
   &gen.write("id", "" | &iteration);
   &gen.write("firstName", "John");
   &gen.write("lastName", "Smith");
   
   REM ** start child address object;
   &gen.writeStartObject("address");
   &gen.write("streetAddress", "21 2nd Street");
   &gen.write("city", "New York");
   &gen.write("state", "NY");
   &gen.write("postalCode", "10021");
   &gen.writeEnd();
   
   REM ** start phone number array;
   &gen.writeStartArray("phoneNumber");
   
   REM ** start home phone object;
   &gen.writeStartObject();
   &gen.write("type", "home");
   &gen.write("number", "212 555-1234");
   &gen.writeEnd();
   
   REM ** start fax number object;
   &gen.writeStartObject();
   &gen.write("type", "fax");
   &gen.write("number", "646 555-4567");
   &gen.writeEnd();
   
   REM ** end array of phone numbers;
   &gen.writeEnd();
   
   REM ** end person/user object;
   &gen.writeEnd();
End-For;


REM ** end array;
&gen.writeEnd();

REM ** cleanup to flush buffers;
&gen.close();
&writer.close();

The hard-coded values come directly from the Jakarta generator API documentation. In real life, you would replace these values with database data. I converted numbers to strings to simplify the example to avoid Java Reflection.

Are you interested in parsing rather than generating large JSON files? Check out our post on JSON Stream Parsing.

We teach PeopleTools and PeopleCode tips like this every week! Check out our upcoming course schedule to see what we are offering next! We would love to have you join us. Want to learn at your own pace? Check out our subscriptions and on-demand offerings as well. Or do you have a group you would like to train? Contact us for group and quantity discounts.

Monday, December 18, 2023

JSON Stream Processing with PeopleCode

Our Integration Tools courses emphasize the importance of utilizing PeopleSoft's native JSON support to parse and process JSON. This feature provides excellent functionality for most scenarios, as PeopleSoft's native JsonObject and JsonArray offer fast and efficient JSON processing. They are particularly useful for generating small integration responses or handling tasks like JWT (JSON Web Token) generation.

It's important to note that PeopleSoft's native JSON definitions employ a DOM-based parser, which builds an in-memory JSON structure. While this is effective in many cases, it does require your server to have enough memory to accommodate the entire JSON document without impacting its normal workload. Large files, therefore, can be problematic.

Stream-based parsing provides an alternative approach. With a stream-based parser, events are emitted as they occur, enabling immediate processing of data identified by these events. Once processed, the parser discards the data and proceeds to the next event. For instance, let's consider a scenario where you have a massive array of users. A DOM parser would load the entire array into memory, whereas a stream-based parser would only load one user at a time, allowing you to process the user and then discard it. Stream-based parsers are often more efficient in terms of resource utilization, as they process and discard data right away instead of constructing a traversable in-memory document.

Although PeopleCode itself doesn't include a stream-based parser, it does include the Java Jakarta stream-based JSON parser, which we can leverage through PeopleSoft's built-in Java support. Let me share an example with you. For testing purposes, I obtained a sample JSON file from https://jsonplaceholder.typicode.com/users. This file is small and perfect for testing. I then incorporated the following PeopleCode into an App Engine step, which you can run locally through App Designer:

REM ** JavaDoc: https://jakarta.ee/specifications/jsonp/2.0/apidocs/jakarta.json/jakarta/json/stream/jsonparser;
REM ** Data: https://jsonplaceholder.typicode.com/users;
Local JavaObject &Json = GetJavaClass("jakarta.json.Json");
Local JavaObject &reader = CreateJavaObject("java.io.FileReader", "C:\temp\users.json");
Local JavaObject &parser = &Json.createParser(&reader);

Local string &email;

While (&parser.hasNext())
   Local JavaObject &next = &parser.next();
   
   If (&next.equals(&next.START_OBJECT)) Then
      
      Local JavaObject &user = &parser.getObject();
      
      REM ** do something with the object;
      &email = &user.getString("email");
      MessageBox(0, "", 0, 0, "email: %1", &email);
   End-If;
End-While;

Here is some sample output generated by this short App Engine program:

email: Sincere@april.biz (0,0)
 Message Set Number: 0
 Message Number: 0
 Message Reason: email: Sincere@april.biz (0,0) (0,0)

email: Shanna@melissa.tv (0,0)
 Message Set Number: 0
 Message Number: 0
 Message Reason: email: Shanna@melissa.tv (0,0) (0,0)

email: Nathan@yesenia.net (0,0)
 Message Set Number: 0
 Message Number: 0
 Message Reason: email: Nathan@yesenia.net (0,0) (0,0)

email: Julianne.OConner@kory.org (0,0)
 Message Set Number: 0
 Message Number: 0
 Message Reason: email: Julianne.OConner@kory.org (0,0) (0,0)

Are you interested in learning more about PeopleSoft Integration, PeopleTools, Or PeopleCode? If so, check out our subscriptions, on-demand, and upcoming course schedule. We would love to have you join us!

Monday, December 04, 2023

Customization Versus Configuration

During PeopleSoft Reconnect 2023, I posted a LinkedIn poll asking if Event Mapping is a Customization or a Configuration. Here is the final tally:



To apply maintenance faster and more often, we must reduce customizations, and configurations are the best alternative to customization. But how would you define a customization? How about a configuration?

Customization

My friend Graham Smith got it right in the poll comments when he said, "The definition of customisation is the changing of a delivered object." 100% correct. If you change a delivered definition, that change will appear in a compare report. If it appears in a compare report, you must analyze and retrofit that change. It is this "analyze and retrofit" effort that delays and even derails Get-current projects. But what if that "change" to a delivered definition happens at runtime through injection rather than design time through a direct code change? Is it still called a customization if the change happens at runtime?

Configuration

This is where it gets tricky. Are Event Mapping, Drop Zones, and Page and Field Configurator configurations? They certainly facilitate configuration. But true configuration is an application-specific construct. Take data masking as an example. We can mask data through Event Mapping and Page and Field Configurator. Masking in this manner requires code that identifies component buffer fields. What if PeopleSoft removes those fields from the component buffer? Our solution would then fail. We might consider this a configuration because we didn't change Oracle-delivered code at design time. Alternatively, we could call it a customization because we changed Oracle-delivered code at runtime.

A true data masking configuration alternative is the Data Privacy Framework, which allows analysts to configure data masking across the application.

One way to think of it is, "who is responsible for fixing the solution if it breaks?" If the answer is Oracle, then it is a configuration. If the answer is you, then it may be a customization.

Isolated Customization

Think of an Isolated Customization as if you isolated your customization from Oracle's delivered codebase and configured PeopleSoft to reinject your customization at runtime. This describes Event Mapping, Drop Zones, and Page and Field Configurator. The value is that your customization no longer shows in a compare report because you have not modified Oracle's delivered code... or have you? Does the point of injection matter? Put another way, does it change the classification if you inject your modification at design time or runtime?

With the introduction of Event Mapping we also might need to see customization as "additional (custom) code to the delivered code." -- Malik Chagani


So here are my tests to help categorize items:

  • Does it show on a compare report? Customization
  • If it breaks, is it my responsibility to fix it? Customization or Isolated Customization
  • If it breaks, is it Oracle's responsibility to fix it? Configuration

Customization, Isolated Customization, or Configuration... why does it matter?
  • Customizations appear on compare reports. We must analyze and retrofit every customization. It is this effort that stalls Get-Current projects.
  • Isolated Customizations don't show on compare reports but have the potential to break when applying maintenance. We must analyze and retrofit Isolated Customizations when they are broken. The challenge with Isolated Customizations is finding them since they don't appear on compare reports.
  • Configurations are not supposed to impact maintenance. You should be able to get current without touching configurations.
What do you think? How do you handle Customizations, Isolated Customizations, and Configurations? Let us know in the comments!

At JSMpros, we regularly teach a course called Configure, Don't Customize, with hands-on activities showing you how to apply dozens of Configuration and Isolation strategies. The table of contents is available online at jsmpros.com.

Wednesday, November 08, 2023

Base64 Encoding with Emoji

PeopleSoft's Pluggable Encryption Technology (PET) is used to apply base64 encoding. The first step of the base64 algorithm chain is to convert from PeopleSoft Unicode to ASCII. The conversion makes sense since PeopleCode is a Unicode language, and the base64 algorithm is not. But what if you have Unicode characters you want to base64 encode? So I tried the following "Hello World" example with emoji:

&crypto.UpdateData("Hello World 🤔");

Unfortunately, the PET algorithm dropped the Unicode Emoji. Understandable because Emoji is Unicode. So, what alternatives do we have? The good news is my older PL/SQL approach still works. However, my older Java example no longer works. So, for those who want a cross-platform solution, here is an updated PeopleCode/Java code listing. The good news is this version is documented and much simpler!

REM ** What would you like to encode?;
Local string &textToEncode = "Hello World 🤔";

REM ** Pointer to Java encoder;
Local JavaObject &encoder = GetJavaClass("java.util.Base64").getEncoder();

REM ** Be sure to change the character set to match your source;
Local JavaObject &bytes = CreateJavaObject("java.lang.String", &textToEncode).getBytes("UTF-8");

Local string &result = &encoder.encodeToString(&bytes);

REM ** print the result;
MessageBox(0, "", 0, 0, &result);

At JSMpros, we teach advanced PeopleTools concepts such as this all the time! Check out our events page to see what we are offering next, and become a subscriber to get 24x7 access to all of our on-demand videos, activity guides, and code samples!

Wednesday, October 18, 2023

PeopleSoft ReConnect 2023



Join me at PeopleSoft Reconnect 2023 for some amazing sessions covering a variety of PeopleTools Topics:

  • Wednesday, Oct 25, 7:30 AM Pacific: PeopleSoft Integration Strategies
  • Thursday, Oct 26, 8:30 AM Pacific: Getting the Most out of PeopleSoft PeopleTools: Tips and Techniques
  • Thursday, Oct 26, 09:45 AM - 10:45 AM PacificMoving from Customized to Configured, Make it Your PeopleSoft
Be sure to add these sessions to your agenda, and keep an eye on the schedule, as sessions are subject to change. Event details and registration are available on the Quest PeopleSoft Community website.

See you online on Monday, October 23 for Paco's executive session, the first session of the conference!


Wednesday, October 04, 2023

Stop Retrofitting and Analyzing Every Customization!

When applying maintenance, we must retrofit every customization. And we will have to continue analyzing and retrofitting as long as we have customizations. The process involves running Compare Reports to identify changes, analyzing customizations, and copying/pasting our old solutions into Oracle's new code base. Occasionally, we must alter a customization to account for Oracle's changes. My point is that we must touch every single customization identified in a Compare Report. The alternative is modern isolation strategies, such as  Page and Field Configurator, Event Mapping, and Drop Zones. We call these isolated customizations because changes are isolated from Oracle's delivered code base. They don't appear in Compare Reports. But should they still be analyzed? Do they require retrofits? Do they break during maintenance?

Without a Compare Report, how do you find isolated customizations? Do you need to find them? We have been discussing this lack of transparency since Event Mapping was released in PeopleTools 8.55. In fact, we've written SQL to help you locate code that includes Event Mapping that was touched by maintenance. But do we need to review every isolated customization? Here is an example. In 2016, I used Event Mapping to change the appearance of the Addresses Page. Notice the iconography next to the Home and Mailing Address in the following screenshot:


I haven't touched this isolated customization in 7 years. That is 7 years and possibly 21 "Get Currents" without a single care for this Isolated Customization! And then, I applied HCM PUM 46. Here is the result:

Event Mapping Failure


I've Waited 7 years for this to happen! Shouldn't a compare report have caught this? No. That is the point of an isolated customization. Our code is isolated from Oracle's code. Therefore, there is no Compare Report. But as you can see, sometimes our code still breaks after maintenance.

So what is the solution? How do we identify isolated customizations that broke while applying maintenance? Regression Tests. The solution to this challenge is the PTF Regression Test. Each time you create an Event Mapping, Drop Zone, or Page and Field Configurator solution, you should create a corresponding PTF regression test to prove your solution still works. A PTF Regression Test would have caught the error message and instantly failed the test. I would then analyze and retrofit just this one broken solution, not every single system change.

Compare Reports for Event Mapping, Drop Zones, and Page and Field Configurator? Do you really need them? Wouldn't a proper Regression Testing strategy save you hours, even days, of analysis by avoiding report reviews?

At JSMpros, we teach developers and business analysts how to create PTF Regression Tests through our two-day PeopleSoft Test Framework course. Find out when we are offering it next, or learn at your own pace through on-demand!