⭐ Comparing our old user API to new user API

Product Payments and Accounting
,
1

@angus @merefield @Hakanto

In preparation for the payment accounting api epic I’ll summarise the ‘as built’ state of the current user api (first iteration) vs the full scope for user and membership api we have in the full spec (see link in post above).

The ‘memberships’ part of the user-membership api was intended to support the member signup and membership admin user stories:

user-membership
membership-class
share-transaction

@merefield confirms it is only the user / iam functions that have been implemented and there is not yet any ‘membership’ functionality:

There is merely a field that tells you if someone is a member or not. It is not modifiable by a non-admin via the api.

The user-api will record the fact but not handle the process of becoming a member or becoming a non-member.

Before we build anything on memberships I think we should check if we could simplify this even further, or even realize these objects with ‘out of the box’ membership solutions elsewhere. … so I am checking here for you for thoughts and comments before shaping up a payments epic.

At the moment I see three main options:

  1. build it bespoke
  2. adapt it in Discourse, using a plugin, plus standard membership features
  3. use a ‘commodity’ proven open source / SAAS hosted solution

1: Build Bespoke

Implement the membership tables / classes as originally specified in golang / postgres as a further iteration, completing the user-membership api as originally envisaged. This is likely to give the best fit to our (current) user stories, but is probably the most effort. It might work well as a back-end for the new signup screens attached to the ID server, but involves a lot of payment service api implementation and maintenance and integration with the Xero accounting API’s

2: Discourse + Plugin

Using a generic Discourse subscription management feature:

This has the advantage of being well-integrated with our community service, but couples us rather tightly to Discourse in future. But: as we are leaning heavily in that direction anyway with our community use of Discourse, the strategic risk of ‘lock/in’ could be acceptable. The bigger question is fitness for purpose and future stability of the plugin. E.g. is the treatment of ‘products’ as access to a private category or group sufficiently robust in this plugin? It’s not like a ‘proper’ e-commerce solution, but it might be close enough to our memberships/shares, top ups and ‘artist circles’ use cases? It is effectively ‘free’ for us as an open-source plugin, and easy to host on our Discourse instance.

3: SAAS / Open Source Component

Implement an e-commerce SAAS web store solution with membership as a ‘product’. Fit? This would provide secure payments integration with proper PCI card handling, security, multiple payment provider integrations. It would also provide a lot of possibilities for more sophisticated store set-up.
Shopify seems to work well with Discourse (Connect Shopify Products to Discourse Groups - Shopify - Pavilion) Cost is initially modest at $360 a year with say 2.5% transaction fees on top of that.

There are other SAAS options available as extensions to our existing payments and accounting solutions, for example:
Xero, who in addition to accounting, provide SAAS invoicing payment solutions for small businesses

See https://developer.xero.com/documentation/api/invoices and https://developer.xero.com/documentation/api-guides/payment-services-integration-with-xero

and also

Stripe: https://stripe.com/use-cases/saas and https://stripe.com/docs/checkout/integration-builder

Mangopay: Crowdfunding | MANGOPAY API Docs

Liberapay: https://liberapay.com/
…which is also open source, and nonprofit.

BackMe: https://backme.org/
…who are an EU focused SEPA patreon alternative that we have talked to previously

There are other open source alternatives like

used OK by webarchitects coop, and maybe

https://www.invoiceninja.com/

Open source accounting solutions like

…have nice invoicing and collections features around the core accounting solution offer, which, like a Xero-based solution, would have the benefit of a pre-integrated accounting ledger. For the Discourse plugin, and other ‘e-commerce’ solutions like Shopify, Stripe invoicing, InvoicePlane and Invoice Ninja, integration with the current Xero back-end accounting system would be required.

Switching to Akaunting might also be possible, or another SAAS accounting solution (many available including Quickbooks, Sage, Freshbooks, Zoho etc) but this would involve additional effort in both selection, familiarisation, integration and migration. I think this isn’t something to take on now - let’s continue to use Xero, and integrate with caution. (Views?)

Tell me if there are any other options to consider here, or if you have any reason to immediately exclude one or more of them. It is of course possible to include a ‘hybrid’ solution, but that might bring further complexity and maintenance burden.

I’m keeping the initial focus on managing our ‘fiscal’ receivables (new memberships and renewals, supporter shares, donations), receipts (the matching payments), payables (credits/reversals and artist payouts) and payments (made) for now.

Later there will also need to be improvements to our internal micropayments and credits, which are only crudely handled by the player accounting at the moment. These will probably be a bespoke solution. I will create a separate component / topic for that.

4 Likes
2

Not trying to push one way or another, but I’m reading this and becoming confused. This seems to be merging two separate topics only related due to the theme of money.

Payments = how the coop inputs / outputs funds - through the streaming platform and the membership fees and paying out to artists.

Accounting = how the coop tracks transactions

Since people could theoretically pay their membership fee using cash or a check and have an admin toggle their membership status to “true”, and the accounting bit still has to happen, these two topics are to me, two totally different things.

For dealing with accounting stuff, that is a question for whoever does the taxes for the coop.

For dealing with payments (paying in and paying out), what’s wrong with Stripe?

Rebuilding Stripe would make Resonate have to deal with PCI compliance, which seems like a lot of overhead.

Downsides to Stripe is that it’s a bit difficult to deal with payouts in some countries (like Mexico) but presumably those issues would exist with or without Stripe.

2 Likes
3

@boopboop Yes Nick is covering a both payments and accounting here as they can be interconnected, as Nick is pointing out I think, but you also make a fair point that it can be good to think of them separately.

To deal with a few individual points

@Nick_M Perhaps we can do a minimal version of this, by storing the external_ids for whatever payment and accounting flow we land on (e.g. storing Stripe ids). As long as we can connect these records easily in the future we can flesh this out further down the line. At the moment, the richer version of member data in the User API feels like a a solution in search of a problem, i.e. we do want to have richer member data stored natively, but largely for purposes that are theoretical at this point (?)

While an interesting idea (and actually something we’ll be implementing at Pavilion soon), this feels like a diversion for Resonate, as eventually we will want member payment integrated into the id server. Moreover the Discourse Subscriptions plugin is essentially just a wrapper of the Stripe API.

@auggod and I have already discussed integrating Stripe into the id server as part of the push to turn off the remaining Wordpress forms i.e. those concerning membership. Stripe’s APIs are relatively easy to implement.

Interesting, but not worth the effort I think. I implemented the flow that article is based on and it can work well, but it’s not well suited to Resonate.

This is the move accounting wise I think. The standard option here would be Xero, but essentially we could use anything that integrates well with payment providers. As we’re currently using Stripe it would at least need to integrate with Stripe.

Concerning Stripe itself, I would point out that their fees are substantial when used at volume. We avoid using Stripe for invoicing at our cooperative for this reason. However at this stage of the game, it’s not worth looking for an alternative

Query whether it would be possible to find a cheaper alternative for consumer-focused transactions for the volumes Resonate has in any event (i.e as @boopboop points out).

@Nick_M I’m thinking this epic will look something like this

  1. Establishing the minimal implementation of stripe on the id server to handle membership payments
  2. Updating the user api to ensure we can connect user records to stripe and accounting records.
  3. Establishing the payments <> accounting flow
4 Likes
4

We do have several supporter shareholders who have complained that they are unable to view their investment and are unable to see where their contributions have landed. Queries about failed payments are similarly difficult to trace and rectify. I’m afraid that some of the basic membership functions of the wordpress membership and profile system are no longer working properly. This isn’t acceptable. Individual statements have to be prepared manually in order to answer these questions. That will not scale in the event of a relaunch, and does not fit well with our regulatory commitment to maintain proper membership registers.

2 Likes
5

Yes, this (and the shopify option) is only suggested as a short-term alternative to building the membership processing described in the user-membership api requirements (as per our rulebook, and current process in wordpress)… just looking at the ‘landscape’ of possible solutions out there to see if there’s anything we might have missed or could learn from.

Adapting a ‘commodity’ membership process that works for others seems a good option to me, even if that plugin isn’t the one. There are lots of possible solutions out there that would seen to match this pattern, especially from Stripe, and perhaps also from Xero. Take a look at this discussion about using the fuller functions of the Stripe API, for example, which includes products, plans, subscriptions and invoices, in this case for an API billing scenario:

In this way we might use this ‘richer’ stripe functionality to do a lot more for us.

image

In the model above, stripe plans (and products) could be used to implement membership classes and all the user membership and share transaction information could be implemented in additional metadata associated with stripe subscriptions.

We use Stripe today as a generic payment for top-ups, and then again via a wordpress /gravity forms / ultimate member solution for memberships. We haven’t used it to implement important things like recurring top-ups or membership renewals. The pattern above would help us to do that, and the flexibility of the ‘plan and product’ objects. It might easier for the Resonate platform to introduce (simple) new products and services by configuration… on a no code / low code basis.

Is it expensive? …only in the longer term.
Does it lock us in to Stripe? …only in the short term… it’s a solid pattern that buys us some time to do something more ‘provider-agnostic’

1 Like
6

Resonate Stripe API - Existing

Yes, we already have stripe feeds implemented in our Xero, but the implementation appears ‘dumb’ at the Stripe end as ‘Stripe Income’ - you have to infer what the payment is for manually by looking at / decoding the payment intent… not a viable effort at scale, and also fails to account properly for failed payments and of course payments via bank, or Wise, for example.

Resonate Stripe API in Development

@auggod has done a great job in refactoring the (existing PHP) stripe API for the tracks API v2, using the ORDERS table:

https://github.com/resonatecoop/tracks-api/blob/main/src/user/payment/api-doc.js
https://github.com/resonatecoop/tracks-api/tree/main/src/user/payment

…so depending on the data passed from that to Stripe, it may improve the intelligibility of the Stripe Data when it comes back in via the Xero accounting feed. There’s a ‘product’ attribute in the payment intent API which in @auggod 's tracks-api code can be set to “credits” (for credit top ups) or to “membership” presumably intended for future re-use for membership payments?

Stripe API futures

Using the Stripe SAAS API’s more fully might bring the additional benefit of a ‘richer’ Stripe to Xero integration feed, where income (streaming credits and subscriptions), currency conversions and tax (particularly VAT) and membership share purchases (these are not ‘income’) are correctly accounted for - automatically. There’s a lot of work involved in doing this correctly at the Resonate end. which is why a proper accounting component, or fuller use of an SAAS service can be attractive.

Xero

However, rather than use the Stripe API directly, I think we should at least consider a short-term use of Xero as a ‘white label’ to create Resonate ‘invoices’ and handle all the associated statement, payments and accounting for us. We could then have Stripe and add a choice of other (cheaper) payment services, especially bank payment, direct debit recurring payments, apple pay (Xero would handle the API’s for and customer payment portal for all these for us). Maybe this is simplistic - it is a small business solution, not intended as an e-commerce solution at scale - but it could be a stepping stone?

Interestingly, In 2017-18 Resonate was invoicing from Xero (presumably for crowdfunding / membership campaigns) - it already has invoices set up, which at that time automatically posted to Stripe or Paypal income, depending on the payment method used to settle them.

image
image859×830 43 KB

Here’s the API:

https://developer.xero.com/documentation/api/accounting/invoices/#overview

…and here is a Golang SDK with examples:

7

Here’s a little more investigation of the SAAS options:

8

Understood. I would note that half of this is “where their contributions have landed”, which is more around the narrative of where the money is being spent, rather than the accounting per se, but point taken.

I would say though that the minimal version of this would be achievable by just storing a couple of foreign keys for that user, rather than modelling the data ourselves.

Using the relevant stripe package (e.g. the npm package for the nodejs based id app), we’d just need the customer id and product id in order to pull their subscription data from the stripe api to display on the relevant profile view.

This is how the Discourse Subscriptions Plugin works. That plugin doesn’t store the product, plan, subscription and invoice data locally, it just stores a few foreign keys, and then when each relevant view is loaded it pulls their data directly from stripe using those keys. It uses stripe itself as a database and model. The easiest way to see this is in the db migrations used to set up the tables the plugin uses

This approach is less performant than storing the data locally, but works fine for light to medium use, which would cover us.

As you say, our initial approach will assume the use of Stripe, however I think we can be even more minimal, by allowing pretty much all the data modelling to be handled by Stripe itself.

(will follow up regarding accounting a bit later in the week)

9

Have a look at the deck, option 2b. I’m proposing we allow pretty much all the data modelling to be done in Xero, not Stripe. There’s even less to model (we make a ‘contact’ and the invoice references to the ‘products’ the user has selected). We’d use our Stripe account indirectly, with a payment button on the Xero-rendered invoice. Everything would automatically reconcile into the correct revenue / equity accounts.

The Golang API example app looks interesting, but presumably we’d have to refactor this to fit in with * GRPC-Gateway and OpenAPIV2 workflow? So maybe instead we should use the GitHub - XeroAPI/xero-node: Xero Node SDK for OAuth 2.0 generated from XeroAPI/Xero-OpenAPI SDK?

10

Isn’t this more steps? If I want to “top up” now, I tap a button and enter my payment info into a stripe-powered form. What happens after is irrelevant to me as a user.

The Xero version would be to pick my selections, view an invoice, tap another button, then pay the invoice.

Perhaps this is better for accounting reasons but it does seem to add another layer of friction to the user?

11

No, we still have to show some form of ‘here’s what you need to pay’ page. We could form the Xero URL and then use the standard rendering for something we would otherwise have had to build. I’m not sure about the return URL though. Xero also handles the item pricing, different currencies and conversion rates as well as the mapping to the correct account. Also you have all the manual reversals / credit functions available to you that you would expect in any sales/service order processing system.

12

OK, I guess I don’t understand the issue enough to opine, since it seems like all the benefits an end-user or investor would want (being able to view a list of transactions, recurring subscriptions, CC management) are possible with Stripe, and having an “invoice step” - to me - would give me pause.

I would of course like to see either an invoice, list of transactions, or some sort of confirmation email that I purchased something, but this seems more of “why are there no automated emails” / “what’s going on with the reporting API” discussion.

Right now I do have a “this is how much you have to pay” in which I pick the level of top-up, before I enter my credit card.

1 Like
13

Yes, you’re right, for top-ups it’s pretty clear what is selected, but there still needs to be some sort of final adding up and totalling, especially if the user is buying a membership share, an annual membership renewal fee, a supporter share and maybe even a donation. At the moment Stripe is invoked in different messy ways (for example, for membership signup via a wordpress form, gravity forms, ultimate member and then stripe…and there are separate custom forms for each of these …each constituting an order/invoice for a product.

In the end, I guess an invoice is just a product offered to someone, with a price, taxes and conditions… and it becomes a contract, with obligations on both sides.

All this sales order processing payments and accounting stuff is pretty standard, and right now I’d be happy to take a lot of it “out of the box”, and be consistent in the way we did it, even it initially the default is perhaps a bit ugly, until we can refine it, and build our own screens that customise more, using the rich API’s available.

Xero have API equivalents for being able to view a list of transactions, recurring subscriptions, CC management) too, without the lock-in to Stripe, which has pretty high fees… this gives us more choice of payments provider without a lot of dev work to do.

I’d rather spend our dev time and resources on a cool player and the playlisting and search API’s that will make us look great. I’d also like to get out of the back office nightmare we live in now!

My biggest concern here is privacy… I’d want to minimise the amount of customer / product information we share with these SAAS services. I think in future, with Community Credentials we may be able to securely package user information and expose far less data to these hungry platforms than they would like us to.

14

Edit: @boopboop @angus

Revisiting this to focus more on the short term. For the co-op right now it is more important to be ready as soon as possible to collect new memberships and renewals asap, with lowest technical risk, never mind about accounting for them. Let’s use the Stripe API we know, and not do an ambitious Payment and Accounting SAAS at this stage.

There are a couple of options on how to do that. One of them involves quite a lot of work in building our ‘checkout’ , with the advantage of better control over the UX flows. The other uses some Stripe SaaS services which reduce the dev effort but do involve a handoff to a Stripe SAAS checkout. In the short term we’ll have to rely on giving worker access to ‘default’ stripe dashboard services to support and account for payments. We would have to allow some Dev time to help with admin and membership queries. We also need to think about extracts for mailshot campaigns to promote renewals.

Where we are now:

Today, Resonate suffers from ‘Dumb’ / manual account mapping and the task of reconciling thousands of individual income transactions - labelled ‘Stripe Income’ or similar with unfriendly ‘Payment Intent’ codes.
To manage membership, there are potentially insecure extracts of membership data to spreadsheets and tedious reconciliation with the payments and accounting treatment of the income / share issues. Non-stripe income from bank payment has much lower transaction fees but involves tedious manual reconciliation.

The user experience in the current WordPress is disjointed and broken - forms are not rendering history of shares and payments in a recognisable ‘statement’. Occasional payment failures are difficult to trace and rectify, and system administrator privileges and valuable developer time is consumed in creating extracts / debugging.

Exchange rates (and VAT rates) are hard coded and are incorrect / out of date. We do not systematically track location / country of income in the accounting.

Payment and Accounting for Short Term - Relaunch

Short Term Option 1

image
image1159×672 147 KB

In this short-term option, we extract the previous year’s (ie only the current) memberships and put them in a temporary membership table. We create a new version of the checkout we used to have on WordPress. We continue to make very simple use of the stripe payment intent API, as we have today, for the credit top ups, but we add metadata to record in the PI ‘what this payment was for’ - to make reconciling the Stripe payment extracts easier later on. (still manual). There is quite a lot of checkout and pricing code to write, test and secure if we are adding all the memberships. For more discussion on this and pro’s and con’s see the deck, slide 5.

Short Term Option 2

image
image1233×717 164 KB

This short term option makes use of the Stripe api (user, product. prices and the checkout) and also uses of a Stripe-hosted checkout page. Pricing and tax (and promotions, discounts etc) are all handled by Stripe. This reduces the amount of code we ahve to write, test, host and secure.
As a by-product, it also makes the stripe extract data better for reconciliation and makes it easier for workers using the Stripe dashboard to find what they need to fix problems in one place. It’s true there is a UX hand-off after the order ‘preview’ but the Stripe pre-built checkout page is widely used and is familiar to consumers, and can be branded/styled/customised.
How would we handle conversion and migration of existing memberships for a ‘renew now’ campaign? We could upload legacy membership details (only the current ones) to the orders and accompanying user tables on Stripe.
See the deck, slide 6.

Mid Term Option

There is a possible mid-term solution that would allow us to bill and account for all our credit top-ups, membership and supporter share products and would be a Xero-specific simplification of @auggod’s stripe api code. It involve the creation of a contact in the Xero SAAS corresponding to a Resonate user, then the creation of an invoice for the products they have ordered (products and pricing all in prior set-up in Xero). We’d use our Stripe account via Xero to make more sense of the payments.
It could provide a wider choice of payment methods for the user, including bank payments and recurring renewals.

However, this SAAS solution is not suitable for a fix to native Resonate credits accounting - no credit transfers yet. Nor does it provide any local ‘cacheing’ of all the data held in the SAAS.

See https://developer.xero.com/documentation/guides/how-to-guides/how-to-integrate-my-pos-system-with-xero/#overview

These API’s are new to us (=risk) and would need some research and investigation. See the deck, slide 7.

Payment and Accounting API Roadmap (post-relaunch)

See the deck, slide 8.

After settling in with the relaunch solution we would evolve a longer term, fuller solution. In addition to handling credit top-ups, membership and supporter share products it would feature a ‘future-proofed’ non-SAAS-specific API gateway. (Such an API gateway could also be a connection point to other coop back end accounting systems, making it easy for them to sell our products on their systems.) It provides better handling of listening credit accounting, including transfers and donations. The user account would be presented from API responses from the SAAS with some ‘cacheing’ and more ‘custom’ rendering to improve on the default SAAS payment model.

1 Like
15

Can you put me in touch with these Supporters?

16

@hakanto has already handled the specific queries that came in members correspondence. I checked share details from the spreadsheet. There’s a thread on basecamp I think.

1 Like
17

@angus @boopboop see update to post above. N

18

Can Stripe/Resonate support currency localization?

19

Yes. https://stripe.com/docs/currencies

Our products are ‘priced’ in € (presentment currency) and our settlement currency is €. Customer credit cards may be in different currencies, and in that case the customer will be charged the fx conversion fee by their cc provider.

We could arrange to ‘present’ in different currencies for users in different locations (check?) and also add settlement accounts in those currencies so that we avoided Stripe fx charges at our end.

I would recommend the co-op sets up a Wise account (if it does not have one already @peter ?) and then we could collect easily in multiple currencies and accept bank transfer income into the same account(s)

20

@zetto.plus recommended this. Also, location is often relevant to content licenses.

Would the check be from the user (cc bank or billing address)? Do devices or routers provide national level location?